Sep 10 11:12:41.522635 ip-10-0-131-79 systemd[1]: Starting Kubernetes Kubelet... Sep 10 11:12:41.936236 ip-10-0-131-79 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 11:12:41.936236 ip-10-0-131-79 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 11:12:41.936236 ip-10-0-131-79 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 11:12:41.936236 ip-10-0-131-79 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 11:12:41.936236 ip-10-0-131-79 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 11:12:41.938289 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.938203 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 11:12:41.943582 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943559 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 11:12:41.943582 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943578 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 11:12:41.943582 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943582 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 11:12:41.943582 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943586 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 11:12:41.943582 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943589 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 11:12:41.943582 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943592 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943595 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943598 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943601 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943603 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943606 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943609 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943611 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943614 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943617 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943619 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943622 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943626 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943630 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943633 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943636 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943639 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943661 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943664 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 11:12:41.943807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943667 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943674 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943679 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943683 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943686 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943690 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943692 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943696 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943699 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943702 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943704 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943707 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943711 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943715 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943717 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943721 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943724 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943727 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943731 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943733 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 11:12:41.944265 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943737 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943740 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943742 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943745 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943748 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943751 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943754 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943757 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943760 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943763 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943765 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943768 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943770 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943773 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943776 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943778 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943781 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943783 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943787 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943789 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 11:12:41.944822 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943792 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943795 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943798 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943800 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943803 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943806 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943809 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943812 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943815 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943817 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943820 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943823 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943826 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943828 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943831 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943833 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943836 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943838 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943841 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 11:12:41.945320 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943846 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943848 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943851 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943854 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943856 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943859 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943863 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943867 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943870 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943873 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943875 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943879 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943881 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943884 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943886 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.943889 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944602 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944608 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944612 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 11:12:41.945791 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944614 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944618 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944621 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944624 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944627 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944630 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944633 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944636 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944638 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944643 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944647 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944650 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944653 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944656 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944659 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944662 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944665 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944667 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944670 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944672 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 11:12:41.946264 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944675 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944677 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944680 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944683 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944686 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944689 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944692 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944695 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944697 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944700 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944703 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944705 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944708 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944711 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944715 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944719 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944722 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944724 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944727 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944730 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 11:12:41.946798 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944732 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944735 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944738 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944742 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944746 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944749 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944752 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944755 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944758 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944760 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944763 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944766 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944768 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944771 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944773 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944776 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944779 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944783 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944785 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 11:12:41.947417 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944788 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944790 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944793 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944795 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944798 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944801 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944803 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944806 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944809 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944811 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944814 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944816 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944819 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944821 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944824 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944827 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944829 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944833 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944836 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944838 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 11:12:41.947886 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944841 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944844 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944847 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944849 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944852 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944854 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944857 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944860 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944862 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944866 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944868 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944871 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944873 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944876 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944878 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944881 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.944884 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945283 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945289 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 11:12:41.948390 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945292 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945313 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945317 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945320 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945323 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945326 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945329 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945332 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945335 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945337 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945340 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945343 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945345 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945348 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945351 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945354 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945357 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945359 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945362 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945364 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 11:12:41.948854 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945367 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945369 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945372 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945375 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945377 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945380 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945383 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945385 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945388 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945390 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945393 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945396 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945398 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945401 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945404 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945406 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945408 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945411 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945414 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945416 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 11:12:41.949386 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945419 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945421 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945424 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945426 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945428 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945431 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945434 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945437 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945439 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945444 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945448 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945450 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945453 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945456 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945459 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945462 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945465 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945467 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945470 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 11:12:41.949880 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945473 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945475 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945477 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945480 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945484 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945488 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945492 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945494 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945497 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945500 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945502 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945505 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945507 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945510 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945513 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945515 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945518 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945522 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945525 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945527 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 11:12:41.950374 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945529 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945532 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945535 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945538 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945540 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945543 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945545 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945548 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945550 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945553 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945556 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945558 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945561 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945564 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945566 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945571 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945573 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.945576 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945654 2560 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945688 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945694 2560 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 11:12:41.950859 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945699 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945703 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945707 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945711 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945715 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945718 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945721 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945725 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945728 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945732 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945735 2560 flags.go:64] FLAG: --cgroup-root="" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945738 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945741 2560 flags.go:64] FLAG: --client-ca-file="" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945749 2560 flags.go:64] FLAG: --cloud-provider="external" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945752 2560 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945757 2560 flags.go:64] FLAG: --cluster-domain="" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945760 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945763 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945766 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945770 2560 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945774 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945777 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945780 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 11:12:41.951389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945783 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945786 2560 flags.go:64] FLAG: --contention-profiling="false" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945789 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945795 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945798 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945801 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945805 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945809 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945812 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945815 2560 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945818 2560 flags.go:64] FLAG: --enable-server="true" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945821 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945825 2560 flags.go:64] FLAG: --event-burst="100" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945828 2560 flags.go:64] FLAG: --event-qps="50" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945831 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945834 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945837 2560 flags.go:64] FLAG: --eviction-hard="" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945841 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945844 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945846 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945850 2560 flags.go:64] FLAG: --eviction-soft="" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945853 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945857 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945861 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945864 2560 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 11:12:41.951949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945867 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945870 2560 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945873 2560 flags.go:64] FLAG: --feature-gates="" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945877 2560 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945880 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945883 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945886 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945889 2560 flags.go:64] FLAG: --healthz-port="10248" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945892 2560 flags.go:64] FLAG: --help="false" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945895 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-131-79.ec2.internal" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945900 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945903 2560 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945905 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945908 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945912 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945915 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945917 2560 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945920 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945923 2560 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945926 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945929 2560 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945932 2560 flags.go:64] FLAG: --kube-reserved="" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945934 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945937 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 11:12:41.952649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945940 2560 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945943 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945946 2560 flags.go:64] FLAG: --lock-file="" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945949 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945951 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945955 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945961 2560 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945964 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945967 2560 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945970 2560 flags.go:64] FLAG: --logging-format="text" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945973 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945976 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945979 2560 flags.go:64] FLAG: --manifest-url="" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945982 2560 flags.go:64] FLAG: --manifest-url-header="" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945986 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945989 2560 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945993 2560 flags.go:64] FLAG: --max-pods="110" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.945996 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946000 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946003 2560 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946006 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946009 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946012 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946015 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946023 2560 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 11:12:41.953236 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946026 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946029 2560 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946032 2560 flags.go:64] FLAG: --pod-cidr="" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946035 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946040 2560 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946043 2560 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946046 2560 flags.go:64] FLAG: --pods-per-core="0" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946049 2560 flags.go:64] FLAG: --port="10250" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946052 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946054 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0efa206430b0d6b1d" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946058 2560 flags.go:64] FLAG: --qos-reserved="" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946060 2560 flags.go:64] FLAG: --read-only-port="10255" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946064 2560 flags.go:64] FLAG: --register-node="true" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946068 2560 flags.go:64] FLAG: --register-with-taints="" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946071 2560 flags.go:64] FLAG: --registry-burst="10" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946080 2560 flags.go:64] FLAG: --registry-qps="5" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946083 2560 flags.go:64] FLAG: --reserved-cpus="" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946086 2560 flags.go:64] FLAG: --reserved-memory="" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946090 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946093 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946095 2560 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946098 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946101 2560 flags.go:64] FLAG: --runonce="false" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946104 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946107 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 11:12:41.953875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946111 2560 flags.go:64] FLAG: --seccomp-default="false" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946114 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946117 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946127 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946131 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946134 2560 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946137 2560 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946140 2560 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946142 2560 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946145 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946148 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946151 2560 flags.go:64] FLAG: --system-cgroups="" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946154 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946160 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946163 2560 flags.go:64] FLAG: --tls-cert-file="" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946165 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946170 2560 flags.go:64] FLAG: --tls-min-version="" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946173 2560 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946175 2560 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946179 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946183 2560 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946186 2560 flags.go:64] FLAG: --v="2" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946190 2560 flags.go:64] FLAG: --version="false" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946194 2560 flags.go:64] FLAG: --vmodule="" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946198 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 11:12:41.954569 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946201 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946327 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946332 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946335 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946338 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946341 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946344 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946349 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946351 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946354 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946356 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946359 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946362 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946364 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946368 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946372 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946376 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946380 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946385 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 11:12:41.955187 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946388 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946392 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946395 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946398 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946400 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946403 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946405 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946408 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946412 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946415 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946418 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946421 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946424 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946427 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946430 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946432 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946435 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946437 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946440 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946443 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 11:12:41.955679 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946446 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946449 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946452 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946454 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946457 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946459 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946462 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946464 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946467 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946470 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946472 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946475 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946478 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946480 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946483 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946485 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946488 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946490 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946494 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 11:12:41.956161 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946498 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946503 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946506 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946508 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946512 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946514 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946517 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946520 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946523 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946525 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946528 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946530 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946533 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946536 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946539 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946542 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946544 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946547 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946549 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946552 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 11:12:41.956685 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946554 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946557 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946561 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946565 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946568 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946570 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946573 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946575 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946578 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946581 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946583 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946586 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946588 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946592 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946595 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946598 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946600 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946603 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946606 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 11:12:41.957176 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946609 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946611 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.946614 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.946619 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.953564 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.953583 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.954231 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.954239 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954321 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954328 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954332 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954336 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954339 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954342 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954345 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 11:12:41.957715 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954348 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954351 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954354 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954356 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954359 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954362 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954364 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954367 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954370 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954372 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954375 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954377 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954380 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954383 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954385 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954388 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954390 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954393 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954395 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954398 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 11:12:41.958096 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954400 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954403 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954407 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954410 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954413 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954415 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954418 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954420 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954423 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954425 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954428 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954430 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954433 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954437 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954442 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954446 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954449 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954451 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954454 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 11:12:41.958633 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954457 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954460 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954462 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954465 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954467 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954470 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954472 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954475 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954478 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954480 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954483 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954485 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954488 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954490 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954493 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954496 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954499 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954502 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954505 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 11:12:41.959094 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954507 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954510 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954512 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954515 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954517 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954520 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954522 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954524 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954527 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954529 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954531 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954534 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954536 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954539 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954541 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954543 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954546 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954548 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954551 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954553 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 11:12:41.959641 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954557 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954561 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954563 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954566 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954569 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954571 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954575 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954578 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954581 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954585 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954588 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954591 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954594 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954596 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 11:12:41.960148 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.954602 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954717 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954722 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954725 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954727 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954730 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954733 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954735 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954738 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954741 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954743 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954746 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954748 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954751 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954753 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954756 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954758 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954761 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954764 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954767 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 11:12:41.960518 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954769 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954771 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954774 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954776 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954779 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954781 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954784 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954786 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954789 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954792 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954795 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954797 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954799 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954802 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954804 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954807 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954809 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954812 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954814 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954817 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 11:12:41.960997 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954819 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954822 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954824 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954827 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954829 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954832 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954834 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954837 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954839 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954842 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954845 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954849 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954852 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954855 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954858 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954861 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954863 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954866 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954868 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954871 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 11:12:41.961495 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954873 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954876 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954884 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954886 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954889 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954892 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954894 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954897 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954899 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954902 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954904 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954908 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954911 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954914 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954917 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954920 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954922 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954925 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954927 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954930 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 11:12:41.961982 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954932 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954935 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954938 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954941 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954943 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954946 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954948 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954951 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954953 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954956 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954958 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954961 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954965 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954968 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954970 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954974 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954976 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954979 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954982 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 11:12:41.962504 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:41.954984 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 11:12:41.962956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.954989 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 11:12:41.962956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.955120 2560 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 11:12:41.962956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.957243 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 11:12:41.962956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.958138 2560 server.go:1019] "Starting client certificate rotation" Sep 10 11:12:41.962956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.958235 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 11:12:41.962956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.958293 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 11:12:41.962956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.961612 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 11:12:41.965251 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.965230 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 11:12:41.976940 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.976917 2560 log.go:25] "Validated CRI v1 runtime API" Sep 10 11:12:41.979777 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.979764 2560 log.go:25] "Validated CRI v1 image API" Sep 10 11:12:41.981056 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.981034 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 11:12:41.985155 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.985126 2560 fs.go:135] Filesystem UUIDs: map[66766e34-728f-48ed-a216-1c8e29f407aa:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a175500f-3bfb-43b9-b073-9004e388d690:/dev/nvme0n1p4] Sep 10 11:12:41.985225 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.985153 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 11:12:41.987182 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.987164 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 11:12:41.990623 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.990506 2560 manager.go:216] Machine: {Timestamp:2026-09-10 11:12:41.989193515 +0000 UTC m=+0.348249712 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111276 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec222c42f664b8ac605c0dd35d817657 SystemUUID:ec222c42-f664-b8ac-605c-0dd35d817657 BootID:e7e4297a-2924-4494-9627-3cd86bd32095 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:02:b0:12:dd:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:02:b0:12:dd:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:46:3b:23:a6:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 11:12:41.990623 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.990620 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 11:12:41.990760 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.990748 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 11:12:41.993955 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.993924 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 11:12:41.994134 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.993957 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-79.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 11:12:41.995174 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.995161 2560 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 11:12:41.995241 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.995182 2560 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 11:12:41.995241 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.995194 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 11:12:41.995862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.995850 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 11:12:41.997015 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:41.997003 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 11:12:42.000622 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.000609 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 10 11:12:42.000766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.000755 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 11:12:42.000826 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.000796 2560 kubelet.go:486] "Attempting to sync node with API server" Sep 10 11:12:42.000826 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.000811 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 11:12:42.000913 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.000833 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 11:12:42.000913 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.000847 2560 kubelet.go:398] "Adding apiserver pod source" Sep 10 11:12:42.000913 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.000861 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 11:12:42.001275 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.001261 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 11:12:42.001364 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.001285 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 11:12:42.003560 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.003544 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 11:12:42.005759 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.005745 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 11:12:42.005802 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.005775 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 11:12:42.005975 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.005965 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 11:12:42.006005 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.005981 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 11:12:42.006005 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.005989 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 11:12:42.006005 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.005999 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 11:12:42.006088 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006010 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 11:12:42.006088 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006015 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 11:12:42.006653 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006643 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 11:12:42.006653 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006654 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 11:12:42.006720 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006662 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 11:12:42.006720 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006668 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 11:12:42.006720 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006676 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 11:12:42.006720 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.006684 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 11:12:42.007592 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.007582 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 11:12:42.007625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.007598 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 11:12:42.011564 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.011534 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 11:12:42.012329 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.011959 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-79.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 11:12:42.012438 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.012194 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 11:12:42.012438 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.012314 2560 server.go:1268] "Started kubelet" Sep 10 11:12:42.012678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.012597 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 11:12:42.012752 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.012734 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 11:12:42.012782 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.012692 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 11:12:42.013586 ip-10-0-131-79 systemd[1]: Started Kubernetes Kubelet. Sep 10 11:12:42.014661 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.014641 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 11:12:42.019696 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.019678 2560 server.go:310] "Adding debug handlers to kubelet server" Sep 10 11:12:42.019926 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.019908 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 11:12:42.021241 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.021226 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2whtd" Sep 10 11:12:42.022231 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.022215 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 11:12:42.022790 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.022774 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 11:12:42.022886 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.021872 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-79.ec2.internal.18d3f1467b255f80 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-79.ec2.internal,UID:ip-10-0-131-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-79.ec2.internal,},FirstTimestamp:2026-09-10 11:12:42.012270464 +0000 UTC m=+0.371326662,LastTimestamp:2026-09-10 11:12:42.012270464 +0000 UTC m=+0.371326662,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-79.ec2.internal,}" Sep 10 11:12:42.023033 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.023014 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.023421 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.023406 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 11:12:42.023421 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.023422 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 11:12:42.023540 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.023406 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 11:12:42.023540 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.023505 2560 reconstruct.go:97] "Volume reconstruction finished" Sep 10 11:12:42.023540 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.023514 2560 reconciler.go:29] "Reconciler: start to sync state" Sep 10 11:12:42.023771 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.023757 2560 factory.go:55] Registering systemd factory Sep 10 11:12:42.023771 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.023772 2560 factory.go:223] Registration of the systemd container factory successfully Sep 10 11:12:42.024029 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.024014 2560 factory.go:153] Registering CRI-O factory Sep 10 11:12:42.024088 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.024031 2560 factory.go:223] Registration of the crio container factory successfully Sep 10 11:12:42.024149 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.024128 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 11:12:42.024198 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.024149 2560 factory.go:103] Registering Raw factory Sep 10 11:12:42.024198 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.024167 2560 manager.go:1196] Started watching for new ooms in manager Sep 10 11:12:42.024650 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.024637 2560 manager.go:349] Starting recovery of all containers Sep 10 11:12:42.030721 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.030698 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2whtd" Sep 10 11:12:42.033620 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.033560 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 11:12:42.033900 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.033847 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 11:12:42.035315 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.035289 2560 manager.go:354] Recovery completed Sep 10 11:12:42.042877 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.042862 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 11:12:42.044818 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.044798 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientMemory" Sep 10 11:12:42.044904 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.044836 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 11:12:42.044904 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.044851 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientPID" Sep 10 11:12:42.045513 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.045492 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 11:12:42.045513 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.045501 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 11:12:42.045513 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.045513 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 11:12:42.045635 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.045519 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 10 11:12:42.046313 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.046287 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientMemory" Sep 10 11:12:42.046439 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.046428 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 11:12:42.046476 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.046444 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientPID" Sep 10 11:12:42.047535 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.047523 2560 policy_none.go:49] "None policy: Start" Sep 10 11:12:42.047579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.047541 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 11:12:42.047579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.047554 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 11:12:42.049547 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.049535 2560 policy_none.go:47] "Start" Sep 10 11:12:42.086995 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.086979 2560 manager.go:337] "Starting Device Plugin manager" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.087028 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.087045 2560 server.go:87] "Starting device plugin registration server" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.087321 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.087336 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.087463 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.087545 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.087560 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.088323 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 11:12:42.109675 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.088370 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.164171 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.164126 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 11:12:42.165487 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.165466 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 11:12:42.165587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.165541 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 11:12:42.165587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.165572 2560 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 11:12:42.165680 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.165621 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 11:12:42.168609 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.168585 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 11:12:42.187927 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.187828 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 11:12:42.191501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.191483 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientMemory" Sep 10 11:12:42.191598 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.191514 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 11:12:42.191598 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.191523 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientPID" Sep 10 11:12:42.191598 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.191546 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.199774 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.199757 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.199827 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.199782 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-79.ec2.internal\": node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.225148 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.225118 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.266145 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.266106 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal"] Sep 10 11:12:42.266328 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.266192 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 11:12:42.267778 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.267762 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientMemory" Sep 10 11:12:42.267878 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.267796 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 11:12:42.267878 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.267809 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientPID" Sep 10 11:12:42.270053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270036 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 11:12:42.270170 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270155 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.270214 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270184 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 11:12:42.270817 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270797 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientMemory" Sep 10 11:12:42.270903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270829 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 11:12:42.270903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270800 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientMemory" Sep 10 11:12:42.270903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270847 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientPID" Sep 10 11:12:42.270903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270866 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 11:12:42.270903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.270878 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientPID" Sep 10 11:12:42.272893 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.272878 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.272955 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.272900 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 11:12:42.273648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.273632 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientMemory" Sep 10 11:12:42.273700 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.273665 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 11:12:42.273700 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.273683 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeHasSufficientPID" Sep 10 11:12:42.306205 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.306178 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-79.ec2.internal\" not found" node="ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.310892 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.310867 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-79.ec2.internal\" not found" node="ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.326136 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.326106 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.424907 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.424872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9576aa1f689d595f1c0703cd66e9d28-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal\" (UID: \"f9576aa1f689d595f1c0703cd66e9d28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.424907 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.424902 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9576aa1f689d595f1c0703cd66e9d28-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal\" (UID: \"f9576aa1f689d595f1c0703cd66e9d28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.425100 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.424918 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6dc34d7642815f28f29ef9663833c3-config\") pod \"kube-apiserver-proxy-ip-10-0-131-79.ec2.internal\" (UID: \"fe6dc34d7642815f28f29ef9663833c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.426976 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.426958 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.525186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.525157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9576aa1f689d595f1c0703cd66e9d28-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal\" (UID: \"f9576aa1f689d595f1c0703cd66e9d28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.525186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.525190 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6dc34d7642815f28f29ef9663833c3-config\") pod \"kube-apiserver-proxy-ip-10-0-131-79.ec2.internal\" (UID: \"fe6dc34d7642815f28f29ef9663833c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.525404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.525208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9576aa1f689d595f1c0703cd66e9d28-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal\" (UID: \"f9576aa1f689d595f1c0703cd66e9d28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.525404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.525279 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9576aa1f689d595f1c0703cd66e9d28-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal\" (UID: \"f9576aa1f689d595f1c0703cd66e9d28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.525404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.525296 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6dc34d7642815f28f29ef9663833c3-config\") pod \"kube-apiserver-proxy-ip-10-0-131-79.ec2.internal\" (UID: \"fe6dc34d7642815f28f29ef9663833c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.525404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.525366 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f9576aa1f689d595f1c0703cd66e9d28-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal\" (UID: \"f9576aa1f689d595f1c0703cd66e9d28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.527420 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.527401 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.614266 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.614220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.614375 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.614287 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" Sep 10 11:12:42.628234 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.628076 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.728338 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.728278 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.830348 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.830248 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.931448 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:42.931412 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:42.959698 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.959673 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 11:12:42.960121 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:42.959816 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 11:12:43.032022 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:43.031990 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-79.ec2.internal\" not found" Sep 10 11:12:43.033936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.033910 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 11:07:42 +0000 UTC" deadline="2028-02-06 22:04:00.096607633 +0000 UTC" Sep 10 11:12:43.033992 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.033936 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12346h51m17.062674862s" Sep 10 11:12:43.048190 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.048169 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 11:12:43.093050 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.092988 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 11:12:43.125021 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.124968 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" Sep 10 11:12:43.138668 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.138649 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 11:12:43.139890 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.139876 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" Sep 10 11:12:43.149174 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.149154 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 11:12:43.486497 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:43.486460 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9576aa1f689d595f1c0703cd66e9d28.slice/crio-d79cd235d3dbd22962ff7ce6eb1faf4354e39df69f6f73e34dade2b881a449c5 WatchSource:0}: Error finding container d79cd235d3dbd22962ff7ce6eb1faf4354e39df69f6f73e34dade2b881a449c5: Status 404 returned error can't find the container with id d79cd235d3dbd22962ff7ce6eb1faf4354e39df69f6f73e34dade2b881a449c5 Sep 10 11:12:43.486736 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:43.486720 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6dc34d7642815f28f29ef9663833c3.slice/crio-2b1813f22119486221cefc6e49897c195aa33061f08ee1f4fcc56927284b36bf WatchSource:0}: Error finding container 2b1813f22119486221cefc6e49897c195aa33061f08ee1f4fcc56927284b36bf: Status 404 returned error can't find the container with id 2b1813f22119486221cefc6e49897c195aa33061f08ee1f4fcc56927284b36bf Sep 10 11:12:43.491586 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.491555 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 11:12:43.583350 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:43.583299 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 11:12:44.003668 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.003627 2560 apiserver.go:52] "Watching apiserver" Sep 10 11:12:44.014292 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.014260 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 11:12:44.015500 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.015471 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dsmzj","kube-system/konnectivity-agent-ppb4s","openshift-dns/node-resolver-pqv7s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal","openshift-multus/multus-52mbp","openshift-multus/multus-additional-cni-plugins-244zd","openshift-multus/network-metrics-daemon-r42wc","openshift-network-diagnostics/network-check-target-9tdqv","kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57","openshift-cluster-node-tuning-operator/tuned-4qq68","openshift-image-registry/node-ca-mt6pz","openshift-network-operator/iptables-alerter-hf4jq","openshift-ovn-kubernetes/ovnkube-node-v4578"] Sep 10 11:12:44.018266 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.018241 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.020690 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.020661 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.022785 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.022753 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 11:12:44.023281 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.023258 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.024783 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.024744 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 11:12:44.024951 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.024911 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 11:12:44.025007 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.024968 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 11:12:44.025169 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.025149 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jb62z\"" Sep 10 11:12:44.025240 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.025209 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 11:12:44.025368 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.025345 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 11:12:44.026057 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.025696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.026057 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.025885 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 11:12:44.026057 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.025898 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l8cfw\"" Sep 10 11:12:44.028351 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.028322 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 11:12:44.028556 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.028537 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.030777 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.030757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.031663 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.031641 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 11:12:44.033564 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.033538 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 11:12:44.033682 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.033659 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fd952\"" Sep 10 11:12:44.033842 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.033821 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-npfpd\"" Sep 10 11:12:44.034034 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.034012 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 11:12:44.034086 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.034058 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 11:12:44.034176 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.034155 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-khdgj\"" Sep 10 11:12:44.036054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.034781 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 11:12:44.036054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.034997 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 11:12:44.036054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.035096 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 11:12:44.036054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.035204 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 11:12:44.036054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.035399 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 11:12:44.036054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.035467 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 11:12:44.036054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.035974 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 11:12:44.036808 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.036789 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2v2kr\"" Sep 10 11:12:44.036970 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.036953 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 11:12:44.037675 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.037654 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 11:12:44.037947 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.037923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-etc-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.037958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-cni-netd\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.037977 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-run-netns\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.037998 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038027 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.038272 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038055 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:44.038272 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-systemd-units\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038272 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-slash\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038272 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038117 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-ovn\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038272 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.038135 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:44.038272 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038143 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-registration-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.038272 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038206 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-sys-fs\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3997314-3e5b-49cf-b445-291aad983581-host-slash\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-systemd\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038491 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-log-socket\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038526 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-cni-bin\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-env-overrides\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvhk7\" (UniqueName: \"kubernetes.io/projected/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-kube-api-access-bvhk7\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038580 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-device-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhwqp\" (UniqueName: \"kubernetes.io/projected/a3997314-3e5b-49cf-b445-291aad983581-kube-api-access-fhwqp\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.038648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-var-lib-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2b64fd3-a514-4a75-acec-78a1678d4044-tmp-dir\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038676 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovnkube-config\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038749 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovn-node-metrics-cert\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-socket-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038811 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rhjv\" (UniqueName: \"kubernetes.io/projected/12e60f7f-5b9b-43d7-a097-a3d80233ad17-kube-api-access-8rhjv\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2b64fd3-a514-4a75-acec-78a1678d4044-hosts-file\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h96lj\" (UniqueName: \"kubernetes.io/projected/b2b64fd3-a514-4a75-acec-78a1678d4044-kube-api-access-h96lj\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038867 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-kubelet\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-node-log\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3997314-3e5b-49cf-b445-291aad983581-iptables-alerter-script\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.039058 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.038960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovnkube-script-lib\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.040409 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.040387 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:44.040497 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.040473 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:44.041609 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.041582 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 11:12:44.044172 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.042940 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.045557 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.045465 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 11:12:44.045684 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.045667 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 11:12:44.046085 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.045868 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vxk57\"" Sep 10 11:12:44.046085 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.045948 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.046085 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.046023 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:44.048267 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.048246 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.050829 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.050810 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.051286 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.051266 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 11:12:44.051839 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.051522 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 11:12:44.051839 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.051597 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bdrcp\"" Sep 10 11:12:44.054818 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.054796 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 11:12:44.055684 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.055658 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 11:12:44.056018 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.056002 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f29r7\"" Sep 10 11:12:44.058687 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.058029 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 11:12:44.071959 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.071933 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cktdh" Sep 10 11:12:44.079915 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.078799 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cktdh" Sep 10 11:12:44.125935 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.125884 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 11:12:44.139241 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-systemd\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139257 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-cni-bin\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-system-cni-dir\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.139412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139339 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-cni-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.139412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc0914eb-81b1-41b1-8553-83522073557c-cni-binary-copy\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.139412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-tmp\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.139620 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139500 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-var-lib-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139620 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139551 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-systemd\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139620 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139591 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-cni-bin\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139710 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-var-lib-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139710 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139689 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-cni-bin\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.139774 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-kubelet\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.139774 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-kubelet-config\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.139841 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139784 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139841 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovn-node-metrics-cert\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.139903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139865 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-socket-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.139903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139890 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-k8s-cni-cncf-io\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.139996 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139924 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-netns\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.139996 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-var-lib-kubelet\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.139996 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.139985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2b64fd3-a514-4a75-acec-78a1678d4044-hosts-file\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.140089 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140015 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h96lj\" (UniqueName: \"kubernetes.io/projected/b2b64fd3-a514-4a75-acec-78a1678d4044-kube-api-access-h96lj\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.140089 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140044 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-kubelet\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.140089 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.140181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140095 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:44.140181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-run\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.140181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140144 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovnkube-script-lib\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.140181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140174 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-cnibin\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.140300 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-cni-netd\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.140300 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140224 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh965\" (UniqueName: \"kubernetes.io/projected/bc0914eb-81b1-41b1-8553-83522073557c-kube-api-access-qh965\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.140300 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:44.140497 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.140497 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-cnibin\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.140497 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-registration-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.140497 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-sys-fs\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.140497 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-hostroot\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-systemd\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140557 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-log-socket\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-env-overrides\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvhk7\" (UniqueName: \"kubernetes.io/projected/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-kube-api-access-bvhk7\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-device-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140672 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p98p9\" (UniqueName: \"kubernetes.io/projected/eff84843-6d59-4eaf-ac58-1740f3ce992f-kube-api-access-p98p9\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-system-cni-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.140746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-cni-multus\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140767 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-etc-kubernetes\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwqp\" (UniqueName: \"kubernetes.io/projected/a3997314-3e5b-49cf-b445-291aad983581-kube-api-access-fhwqp\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140818 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-modprobe-d\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-host\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1b5e602-0217-49c5-8c46-502120dff851-host\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140897 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1b5e602-0217-49c5-8c46-502120dff851-serviceca\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.140936 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw54m\" (UniqueName: \"kubernetes.io/projected/a1b5e602-0217-49c5-8c46-502120dff851-kube-api-access-bw54m\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141010 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2b64fd3-a514-4a75-acec-78a1678d4044-tmp-dir\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovnkube-config\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141083 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rhjv\" (UniqueName: \"kubernetes.io/projected/12e60f7f-5b9b-43d7-a097-a3d80233ad17-kube-api-access-8rhjv\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141111 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-cni-binary-copy\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-conf-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.141185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141165 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z8md\" (UniqueName: \"kubernetes.io/projected/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-kube-api-access-7z8md\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141197 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-node-log\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141254 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141281 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141338 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aecc4dcd-2c08-453f-8202-1f571e23862a-konnectivity-ca\") pod \"konnectivity-agent-ppb4s\" (UID: \"aecc4dcd-2c08-453f-8202-1f571e23862a\") " pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3997314-3e5b-49cf-b445-291aad983581-iptables-alerter-script\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141420 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aecc4dcd-2c08-453f-8202-1f571e23862a-agent-certs\") pod \"konnectivity-agent-ppb4s\" (UID: \"aecc4dcd-2c08-453f-8202-1f571e23862a\") " pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141462 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysctl-conf\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-lib-modules\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-tuned\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141604 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-etc-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8qbz\" (UniqueName: \"kubernetes.io/projected/f4f8581c-3527-40bb-89f6-b51b45a0d4af-kube-api-access-f8qbz\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141679 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-run-netns\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141737 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc0914eb-81b1-41b1-8553-83522073557c-multus-daemon-config\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.141862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141762 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-dbus\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-kubernetes\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysctl-d\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141866 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-sys\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141899 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-systemd-units\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-slash\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.141973 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-ovn\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.142000 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-socket-dir-parent\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.142030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-multus-certs\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.142060 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysconfig\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.142089 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3997314-3e5b-49cf-b445-291aad983581-host-slash\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.142192 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3997314-3e5b-49cf-b445-291aad983581-host-slash\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.142283 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.142656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.142338 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.143365 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143101 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2b64fd3-a514-4a75-acec-78a1678d4044-tmp-dir\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.143365 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143310 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-node-log\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.143501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143369 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-etc-selinux\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.143501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143429 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-kubelet\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.143601 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-cni-netd\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.143659 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143609 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-etc-openvswitch\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.143659 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143634 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.143758 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143683 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-run-netns\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.143758 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-registration-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.143758 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143742 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-sys-fs\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.143906 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143805 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-log-socket\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.143906 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143893 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-systemd-units\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.144094 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143936 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2b64fd3-a514-4a75-acec-78a1678d4044-hosts-file\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.144352 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.143985 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-slash\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.144756 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.144721 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3997314-3e5b-49cf-b445-291aad983581-iptables-alerter-script\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.144862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.144795 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-run-ovn\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.144946 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.144868 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 11:12:44.144946 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.144876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-socket-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.145807 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.145459 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-host-run-ovn-kubernetes\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.145807 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.145466 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovnkube-script-lib\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.145959 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.145943 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovnkube-config\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.146027 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.146005 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-env-overrides\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.146847 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.146221 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/12e60f7f-5b9b-43d7-a097-a3d80233ad17-device-dir\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.152417 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.152392 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-ovn-node-metrics-cert\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.155278 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.155249 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h96lj\" (UniqueName: \"kubernetes.io/projected/b2b64fd3-a514-4a75-acec-78a1678d4044-kube-api-access-h96lj\") pod \"node-resolver-pqv7s\" (UID: \"b2b64fd3-a514-4a75-acec-78a1678d4044\") " pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.155401 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.155317 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvhk7\" (UniqueName: \"kubernetes.io/projected/811e7c53-3cb8-4df3-96d1-e1e823aa63ee-kube-api-access-bvhk7\") pod \"ovnkube-node-v4578\" (UID: \"811e7c53-3cb8-4df3-96d1-e1e823aa63ee\") " pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.155401 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.155326 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhwqp\" (UniqueName: \"kubernetes.io/projected/a3997314-3e5b-49cf-b445-291aad983581-kube-api-access-fhwqp\") pod \"iptables-alerter-hf4jq\" (UID: \"a3997314-3e5b-49cf-b445-291aad983581\") " pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.157195 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.157172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rhjv\" (UniqueName: \"kubernetes.io/projected/12e60f7f-5b9b-43d7-a097-a3d80233ad17-kube-api-access-8rhjv\") pod \"aws-ebs-csi-driver-node-9kk57\" (UID: \"12e60f7f-5b9b-43d7-a097-a3d80233ad17\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.171071 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.171016 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" event={"ID":"fe6dc34d7642815f28f29ef9663833c3","Type":"ContainerStarted","Data":"2b1813f22119486221cefc6e49897c195aa33061f08ee1f4fcc56927284b36bf"} Sep 10 11:12:44.172946 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.172887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" event={"ID":"f9576aa1f689d595f1c0703cd66e9d28","Type":"ContainerStarted","Data":"d79cd235d3dbd22962ff7ce6eb1faf4354e39df69f6f73e34dade2b881a449c5"} Sep 10 11:12:44.238030 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.237997 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 11:12:44.242386 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc0914eb-81b1-41b1-8553-83522073557c-multus-daemon-config\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242543 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-dbus\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.242543 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-kubernetes\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242543 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysctl-d\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242564 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-kubernetes\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysctl-d\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-sys\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-socket-dir-parent\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242672 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-sys\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-multus-certs\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242711 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysconfig\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242720 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-multus-certs\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242721 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-dbus\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242734 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-socket-dir-parent\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242744 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-system-cni-dir\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242770 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysconfig\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242779 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-system-cni-dir\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-cni-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242832 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc0914eb-81b1-41b1-8553-83522073557c-cni-binary-copy\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-tmp\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242892 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-cni-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-cni-bin\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242930 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-kubelet\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242942 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-cni-bin\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-kubelet-config\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242986 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-kubelet\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.242991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.242990 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-k8s-cni-cncf-io\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243023 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-kubelet-config\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-netns\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bc0914eb-81b1-41b1-8553-83522073557c-multus-daemon-config\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243052 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-k8s-cni-cncf-io\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243081 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-run-netns\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243108 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-var-lib-kubelet\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243147 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-var-lib-kubelet\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243177 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-run\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-cnibin\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243253 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh965\" (UniqueName: \"kubernetes.io/projected/bc0914eb-81b1-41b1-8553-83522073557c-kube-api-access-qh965\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243272 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-run\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243281 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243309 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-cnibin\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243337 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-cnibin\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243346 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-hostroot\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.243855 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.243365 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-systemd\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243416 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc0914eb-81b1-41b1-8553-83522073557c-cni-binary-copy\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.243450 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs podName:f4f8581c-3527-40bb-89f6-b51b45a0d4af nodeName:}" failed. No retries permitted until 2026-09-10 11:12:44.743428506 +0000 UTC m=+3.102484706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs") pod "network-metrics-daemon-r42wc" (UID: "f4f8581c-3527-40bb-89f6-b51b45a0d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243476 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-cnibin\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243520 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-hostroot\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p98p9\" (UniqueName: \"kubernetes.io/projected/eff84843-6d59-4eaf-ac58-1740f3ce992f-kube-api-access-p98p9\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.243600 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-system-cni-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243613 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-systemd\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.243655 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret podName:7f5e0a42-7fba-460a-b3d2-67a92ca2c94e nodeName:}" failed. No retries permitted until 2026-09-10 11:12:44.743636811 +0000 UTC m=+3.102692997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret") pod "global-pull-secret-syncer-dsmzj" (UID: "7f5e0a42-7fba-460a-b3d2-67a92ca2c94e") : object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243669 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-system-cni-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243712 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-cni-multus\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-etc-kubernetes\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243755 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.244795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243757 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-host-var-lib-cni-multus\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243774 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-modprobe-d\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243803 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-etc-kubernetes\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243810 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-host\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243849 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1b5e602-0217-49c5-8c46-502120dff851-host\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243870 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-host\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1b5e602-0217-49c5-8c46-502120dff851-serviceca\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw54m\" (UniqueName: \"kubernetes.io/projected/a1b5e602-0217-49c5-8c46-502120dff851-kube-api-access-bw54m\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243931 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1b5e602-0217-49c5-8c46-502120dff851-host\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243873 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-modprobe-d\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-cni-binary-copy\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.243988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-conf-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244013 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z8md\" (UniqueName: \"kubernetes.io/projected/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-kube-api-access-7z8md\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244044 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aecc4dcd-2c08-453f-8202-1f571e23862a-konnectivity-ca\") pod \"konnectivity-agent-ppb4s\" (UID: \"aecc4dcd-2c08-453f-8202-1f571e23862a\") " pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244154 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aecc4dcd-2c08-453f-8202-1f571e23862a-agent-certs\") pod \"konnectivity-agent-ppb4s\" (UID: \"aecc4dcd-2c08-453f-8202-1f571e23862a\") " pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysctl-conf\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.245625 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244212 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-lib-modules\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-tuned\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244264 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8qbz\" (UniqueName: \"kubernetes.io/projected/f4f8581c-3527-40bb-89f6-b51b45a0d4af-kube-api-access-f8qbz\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244294 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1b5e602-0217-49c5-8c46-502120dff851-serviceca\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244375 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc0914eb-81b1-41b1-8553-83522073557c-multus-conf-dir\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eff84843-6d59-4eaf-ac58-1740f3ce992f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244537 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-cni-binary-copy\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244623 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-sysctl-conf\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244662 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-lib-modules\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.244989 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eff84843-6d59-4eaf-ac58-1740f3ce992f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.246508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.245196 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aecc4dcd-2c08-453f-8202-1f571e23862a-konnectivity-ca\") pod \"konnectivity-agent-ppb4s\" (UID: \"aecc4dcd-2c08-453f-8202-1f571e23862a\") " pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.246845 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.246672 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-tmp\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.246974 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.246956 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-etc-tuned\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.247444 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.247419 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aecc4dcd-2c08-453f-8202-1f571e23862a-agent-certs\") pod \"konnectivity-agent-ppb4s\" (UID: \"aecc4dcd-2c08-453f-8202-1f571e23862a\") " pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.260500 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.260384 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh965\" (UniqueName: \"kubernetes.io/projected/bc0914eb-81b1-41b1-8553-83522073557c-kube-api-access-qh965\") pod \"multus-52mbp\" (UID: \"bc0914eb-81b1-41b1-8553-83522073557c\") " pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.261406 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.261384 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 11:12:44.261525 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.261411 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 11:12:44.261525 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.261426 2560 projected.go:196] Error preparing data for projected volume kube-api-access-sk5qj for pod openshift-network-diagnostics/network-check-target-9tdqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:44.261525 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.261482 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj podName:6ca7ad53-128f-4ce9-8dc7-ffa5482323c8 nodeName:}" failed. No retries permitted until 2026-09-10 11:12:44.761465282 +0000 UTC m=+3.120521467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sk5qj" (UniqueName: "kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj") pod "network-check-target-9tdqv" (UID: "6ca7ad53-128f-4ce9-8dc7-ffa5482323c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:44.264448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.263982 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z8md\" (UniqueName: \"kubernetes.io/projected/70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b-kube-api-access-7z8md\") pod \"tuned-4qq68\" (UID: \"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b\") " pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.264448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.264029 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p98p9\" (UniqueName: \"kubernetes.io/projected/eff84843-6d59-4eaf-ac58-1740f3ce992f-kube-api-access-p98p9\") pod \"multus-additional-cni-plugins-244zd\" (UID: \"eff84843-6d59-4eaf-ac58-1740f3ce992f\") " pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.264448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.264404 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw54m\" (UniqueName: \"kubernetes.io/projected/a1b5e602-0217-49c5-8c46-502120dff851-kube-api-access-bw54m\") pod \"node-ca-mt6pz\" (UID: \"a1b5e602-0217-49c5-8c46-502120dff851\") " pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.265322 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.265292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8qbz\" (UniqueName: \"kubernetes.io/projected/f4f8581c-3527-40bb-89f6-b51b45a0d4af-kube-api-access-f8qbz\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:44.334470 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.334164 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" Sep 10 11:12:44.345502 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.345469 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hf4jq" Sep 10 11:12:44.355825 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.355715 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pqv7s" Sep 10 11:12:44.365190 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.362817 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:12:44.371949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.371881 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52mbp" Sep 10 11:12:44.374979 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:44.374868 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b64fd3_a514_4a75_acec_78a1678d4044.slice/crio-05c2e4225fd289dd7be4c61107ce66df3a136bc0e99f8a6d04c48e1560773cb2 WatchSource:0}: Error finding container 05c2e4225fd289dd7be4c61107ce66df3a136bc0e99f8a6d04c48e1560773cb2: Status 404 returned error can't find the container with id 05c2e4225fd289dd7be4c61107ce66df3a136bc0e99f8a6d04c48e1560773cb2 Sep 10 11:12:44.384041 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.384008 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-244zd" Sep 10 11:12:44.386441 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:44.386364 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0914eb_81b1_41b1_8553_83522073557c.slice/crio-18405ef869dc275f67e59f2f627e25b13529962018467e251ae6df0a63df1172 WatchSource:0}: Error finding container 18405ef869dc275f67e59f2f627e25b13529962018467e251ae6df0a63df1172: Status 404 returned error can't find the container with id 18405ef869dc275f67e59f2f627e25b13529962018467e251ae6df0a63df1172 Sep 10 11:12:44.393234 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.393206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:12:44.397528 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:44.397493 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff84843_6d59_4eaf_ac58_1740f3ce992f.slice/crio-dfdb8b4f150cbf3834e71ed74f451a164322206275f0faf2e4144114e0043cde WatchSource:0}: Error finding container dfdb8b4f150cbf3834e71ed74f451a164322206275f0faf2e4144114e0043cde: Status 404 returned error can't find the container with id dfdb8b4f150cbf3834e71ed74f451a164322206275f0faf2e4144114e0043cde Sep 10 11:12:44.401178 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.400879 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4qq68" Sep 10 11:12:44.409780 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:44.406767 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaecc4dcd_2c08_453f_8202_1f571e23862a.slice/crio-15268e65260e1cab0f7499629604b116acb1925e21de6bc576543597836bf3cd WatchSource:0}: Error finding container 15268e65260e1cab0f7499629604b116acb1925e21de6bc576543597836bf3cd: Status 404 returned error can't find the container with id 15268e65260e1cab0f7499629604b116acb1925e21de6bc576543597836bf3cd Sep 10 11:12:44.409780 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.408438 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mt6pz" Sep 10 11:12:44.414068 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:12:44.414041 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b2b1a4_e63a_4fb3_aa44_ebc01c2e103b.slice/crio-4fa3f508770c5d2b5fc92acf10e01b2a71470b82cf9a6d7a9f836215cf7fce80 WatchSource:0}: Error finding container 4fa3f508770c5d2b5fc92acf10e01b2a71470b82cf9a6d7a9f836215cf7fce80: Status 404 returned error can't find the container with id 4fa3f508770c5d2b5fc92acf10e01b2a71470b82cf9a6d7a9f836215cf7fce80 Sep 10 11:12:44.750010 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.749900 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:44.750010 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.750009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:44.750223 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.750163 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:44.750277 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.750232 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs podName:f4f8581c-3527-40bb-89f6-b51b45a0d4af nodeName:}" failed. No retries permitted until 2026-09-10 11:12:45.750214335 +0000 UTC m=+4.109270536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs") pod "network-metrics-daemon-r42wc" (UID: "f4f8581c-3527-40bb-89f6-b51b45a0d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:44.750487 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.750455 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:44.750629 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.750528 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret podName:7f5e0a42-7fba-460a-b3d2-67a92ca2c94e nodeName:}" failed. No retries permitted until 2026-09-10 11:12:45.750509647 +0000 UTC m=+4.109565848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret") pod "global-pull-secret-syncer-dsmzj" (UID: "7f5e0a42-7fba-460a-b3d2-67a92ca2c94e") : object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:44.850770 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:44.850729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:44.850952 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.850927 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 11:12:44.851006 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.850954 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 11:12:44.851006 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.850966 2560 projected.go:196] Error preparing data for projected volume kube-api-access-sk5qj for pod openshift-network-diagnostics/network-check-target-9tdqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:44.851071 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:44.851024 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj podName:6ca7ad53-128f-4ce9-8dc7-ffa5482323c8 nodeName:}" failed. No retries permitted until 2026-09-10 11:12:45.85101036 +0000 UTC m=+4.210066546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk5qj" (UniqueName: "kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj") pod "network-check-target-9tdqv" (UID: "6ca7ad53-128f-4ce9-8dc7-ffa5482323c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:45.082594 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.082481 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 11:07:44 +0000 UTC" deadline="2028-03-28 01:50:47.112256302 +0000 UTC" Sep 10 11:12:45.082594 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.082547 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13550h38m2.029713569s" Sep 10 11:12:45.166636 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.166600 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:45.166853 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.166813 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:45.176961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.176918 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4qq68" event={"ID":"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b","Type":"ContainerStarted","Data":"4fa3f508770c5d2b5fc92acf10e01b2a71470b82cf9a6d7a9f836215cf7fce80"} Sep 10 11:12:45.178816 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.178735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ppb4s" event={"ID":"aecc4dcd-2c08-453f-8202-1f571e23862a","Type":"ContainerStarted","Data":"15268e65260e1cab0f7499629604b116acb1925e21de6bc576543597836bf3cd"} Sep 10 11:12:45.180377 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.180344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerStarted","Data":"dfdb8b4f150cbf3834e71ed74f451a164322206275f0faf2e4144114e0043cde"} Sep 10 11:12:45.182026 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.181997 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pqv7s" event={"ID":"b2b64fd3-a514-4a75-acec-78a1678d4044","Type":"ContainerStarted","Data":"05c2e4225fd289dd7be4c61107ce66df3a136bc0e99f8a6d04c48e1560773cb2"} Sep 10 11:12:45.183655 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.183628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" event={"ID":"12e60f7f-5b9b-43d7-a097-a3d80233ad17","Type":"ContainerStarted","Data":"4768cbf7e4d323cb68f70c2c31d00cbd9782b10b6094b59a5fb4703b13de4e63"} Sep 10 11:12:45.185389 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.185361 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mt6pz" event={"ID":"a1b5e602-0217-49c5-8c46-502120dff851","Type":"ContainerStarted","Data":"6483315454ab0fbe9a15ca333f5792e1690083009976322bf3d0a16c8cccf7c6"} Sep 10 11:12:45.187845 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.187817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52mbp" event={"ID":"bc0914eb-81b1-41b1-8553-83522073557c","Type":"ContainerStarted","Data":"18405ef869dc275f67e59f2f627e25b13529962018467e251ae6df0a63df1172"} Sep 10 11:12:45.189854 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.189826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"5ce9cc19e8fbc578b11167f65d2a493898777fd03fa49defaf7606e8976fb190"} Sep 10 11:12:45.191465 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.191438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hf4jq" event={"ID":"a3997314-3e5b-49cf-b445-291aad983581","Type":"ContainerStarted","Data":"32d231049c03e7ecf5ca07c2a63673105c7b5edff46c076201a2f156d0da53ca"} Sep 10 11:12:45.759258 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.759217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:45.759436 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.759325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:45.759509 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.759465 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:45.759549 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.759529 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs podName:f4f8581c-3527-40bb-89f6-b51b45a0d4af nodeName:}" failed. No retries permitted until 2026-09-10 11:12:47.759510215 +0000 UTC m=+6.118566414 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs") pod "network-metrics-daemon-r42wc" (UID: "f4f8581c-3527-40bb-89f6-b51b45a0d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:45.759646 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.759598 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:45.759646 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.759640 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret podName:7f5e0a42-7fba-460a-b3d2-67a92ca2c94e nodeName:}" failed. No retries permitted until 2026-09-10 11:12:47.759628285 +0000 UTC m=+6.118684475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret") pod "global-pull-secret-syncer-dsmzj" (UID: "7f5e0a42-7fba-460a-b3d2-67a92ca2c94e") : object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:45.860258 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:45.859677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:45.860258 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.859846 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 11:12:45.860258 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.859868 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 11:12:45.860258 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.859880 2560 projected.go:196] Error preparing data for projected volume kube-api-access-sk5qj for pod openshift-network-diagnostics/network-check-target-9tdqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:45.860258 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:45.859929 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj podName:6ca7ad53-128f-4ce9-8dc7-ffa5482323c8 nodeName:}" failed. No retries permitted until 2026-09-10 11:12:47.859912638 +0000 UTC m=+6.218968822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk5qj" (UniqueName: "kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj") pod "network-check-target-9tdqv" (UID: "6ca7ad53-128f-4ce9-8dc7-ffa5482323c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:46.082699 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:46.082666 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 11:07:44 +0000 UTC" deadline="2028-04-22 20:39:16.704124956 +0000 UTC" Sep 10 11:12:46.082699 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:46.082696 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14169h26m30.621431759s" Sep 10 11:12:46.166554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:46.166315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:46.166698 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:46.166657 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:46.166967 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:46.166390 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:46.167089 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:46.167066 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:46.201846 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:46.201806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" event={"ID":"fe6dc34d7642815f28f29ef9663833c3","Type":"ContainerStarted","Data":"2fc6055c24fc18c0899619428f8848d69ce9cc58c5bb81266a6d80807a210e76"} Sep 10 11:12:47.166040 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:47.165948 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:47.166515 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.166090 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:47.779189 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:47.779147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:47.779443 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:47.779218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:47.779443 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.779382 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:47.779579 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.779448 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret podName:7f5e0a42-7fba-460a-b3d2-67a92ca2c94e nodeName:}" failed. No retries permitted until 2026-09-10 11:12:51.779429625 +0000 UTC m=+10.138485823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret") pod "global-pull-secret-syncer-dsmzj" (UID: "7f5e0a42-7fba-460a-b3d2-67a92ca2c94e") : object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:47.779866 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.779838 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:47.779956 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.779884 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs podName:f4f8581c-3527-40bb-89f6-b51b45a0d4af nodeName:}" failed. No retries permitted until 2026-09-10 11:12:51.779870047 +0000 UTC m=+10.138926244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs") pod "network-metrics-daemon-r42wc" (UID: "f4f8581c-3527-40bb-89f6-b51b45a0d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:47.879850 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:47.879812 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:47.880036 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.880017 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 11:12:47.880128 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.880044 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 11:12:47.880128 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.880058 2560 projected.go:196] Error preparing data for projected volume kube-api-access-sk5qj for pod openshift-network-diagnostics/network-check-target-9tdqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:47.880128 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:47.880114 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj podName:6ca7ad53-128f-4ce9-8dc7-ffa5482323c8 nodeName:}" failed. No retries permitted until 2026-09-10 11:12:51.88009634 +0000 UTC m=+10.239152538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk5qj" (UniqueName: "kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj") pod "network-check-target-9tdqv" (UID: "6ca7ad53-128f-4ce9-8dc7-ffa5482323c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:48.167637 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:48.167552 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:48.167637 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:48.167599 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:48.168114 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:48.167706 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:48.168114 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:48.167805 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:49.166691 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:49.166593 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:49.166847 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:49.166744 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:50.166315 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:50.166267 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:50.166768 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:50.166419 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:50.166768 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:50.166269 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:50.166895 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:50.166866 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:50.215359 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:50.215324 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" event={"ID":"f9576aa1f689d595f1c0703cd66e9d28","Type":"ContainerStarted","Data":"c6d9e84af03f19f5c40fa5105e1ac4277222ea327c52a02f6d372d73fc63f48f"} Sep 10 11:12:50.236985 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:50.236922 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-79.ec2.internal" podStartSLOduration=7.236901771 podStartE2EDuration="7.236901771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:12:46.22381139 +0000 UTC m=+4.582867597" watchObservedRunningTime="2026-09-10 11:12:50.236901771 +0000 UTC m=+8.595957976" Sep 10 11:12:51.166823 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:51.166742 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:51.167197 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.166851 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:51.812644 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:51.812591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:51.812817 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:51.812659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:51.812817 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.812766 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:51.812939 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.812830 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret podName:7f5e0a42-7fba-460a-b3d2-67a92ca2c94e nodeName:}" failed. No retries permitted until 2026-09-10 11:12:59.812809258 +0000 UTC m=+18.171865455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret") pod "global-pull-secret-syncer-dsmzj" (UID: "7f5e0a42-7fba-460a-b3d2-67a92ca2c94e") : object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:51.812939 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.812762 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:51.812939 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.812915 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs podName:f4f8581c-3527-40bb-89f6-b51b45a0d4af nodeName:}" failed. No retries permitted until 2026-09-10 11:12:59.812895567 +0000 UTC m=+18.171951774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs") pod "network-metrics-daemon-r42wc" (UID: "f4f8581c-3527-40bb-89f6-b51b45a0d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:51.913842 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:51.913356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:51.913842 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.913557 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 11:12:51.913842 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.913583 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 11:12:51.913842 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.913595 2560 projected.go:196] Error preparing data for projected volume kube-api-access-sk5qj for pod openshift-network-diagnostics/network-check-target-9tdqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:51.913842 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:51.913658 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj podName:6ca7ad53-128f-4ce9-8dc7-ffa5482323c8 nodeName:}" failed. No retries permitted until 2026-09-10 11:12:59.91363864 +0000 UTC m=+18.272694828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk5qj" (UniqueName: "kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj") pod "network-check-target-9tdqv" (UID: "6ca7ad53-128f-4ce9-8dc7-ffa5482323c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:52.167291 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:52.167197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:52.167750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:52.167349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:52.167750 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:52.167372 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:52.167750 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:52.167429 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:53.166476 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:53.166443 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:53.166654 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:53.166582 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:54.166457 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:54.166424 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:54.166915 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:54.166467 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:54.166915 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:54.166691 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:54.166915 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:54.166820 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:55.166083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:55.166042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:55.166253 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:55.166169 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:55.227924 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:55.227890 2560 generic.go:356] "Generic (PLEG): container finished" podID="f9576aa1f689d595f1c0703cd66e9d28" containerID="c6d9e84af03f19f5c40fa5105e1ac4277222ea327c52a02f6d372d73fc63f48f" exitCode=0 Sep 10 11:12:55.228404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:55.227939 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" event={"ID":"f9576aa1f689d595f1c0703cd66e9d28","Type":"ContainerDied","Data":"c6d9e84af03f19f5c40fa5105e1ac4277222ea327c52a02f6d372d73fc63f48f"} Sep 10 11:12:56.165987 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:56.165945 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:56.166178 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:56.166068 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:56.166178 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:56.166130 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:56.166266 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:56.166249 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:57.166076 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:57.166044 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:57.166519 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:57.166166 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:58.165945 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:58.165908 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:58.165945 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:58.165928 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:58.166393 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:58.166060 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:12:58.166393 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:58.166180 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:12:59.165930 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:59.165890 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:59.166143 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.166022 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:12:59.874456 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:59.874420 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:12:59.874456 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:59.874468 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:12:59.874695 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.874603 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:59.874695 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.874618 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:59.874695 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.874665 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret podName:7f5e0a42-7fba-460a-b3d2-67a92ca2c94e nodeName:}" failed. No retries permitted until 2026-09-10 11:13:15.874649388 +0000 UTC m=+34.233705574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret") pod "global-pull-secret-syncer-dsmzj" (UID: "7f5e0a42-7fba-460a-b3d2-67a92ca2c94e") : object "kube-system"/"original-pull-secret" not registered Sep 10 11:12:59.874695 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.874686 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs podName:f4f8581c-3527-40bb-89f6-b51b45a0d4af nodeName:}" failed. No retries permitted until 2026-09-10 11:13:15.874675703 +0000 UTC m=+34.233731891 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs") pod "network-metrics-daemon-r42wc" (UID: "f4f8581c-3527-40bb-89f6-b51b45a0d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:12:59.974892 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:12:59.974857 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:12:59.975066 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.975040 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 11:12:59.975132 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.975071 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 11:12:59.975132 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.975087 2560 projected.go:196] Error preparing data for projected volume kube-api-access-sk5qj for pod openshift-network-diagnostics/network-check-target-9tdqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:12:59.975205 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:12:59.975152 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj podName:6ca7ad53-128f-4ce9-8dc7-ffa5482323c8 nodeName:}" failed. No retries permitted until 2026-09-10 11:13:15.975132736 +0000 UTC m=+34.334188941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk5qj" (UniqueName: "kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj") pod "network-check-target-9tdqv" (UID: "6ca7ad53-128f-4ce9-8dc7-ffa5482323c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:13:00.166497 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:00.166467 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:00.166907 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:00.166470 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:00.166907 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:00.166613 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:00.166907 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:00.166706 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:01.166672 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:01.166634 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:01.167103 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:01.166764 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:02.166832 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:02.166796 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:02.167211 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:02.166889 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:02.167211 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:02.166978 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:02.167211 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:02.167090 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:03.166282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.166088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:03.166433 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:03.166367 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:03.245594 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.245483 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pqv7s" event={"ID":"b2b64fd3-a514-4a75-acec-78a1678d4044","Type":"ContainerStarted","Data":"1633d18a3223acf44e504335b805b83a76a33aa92797d37b33817325dbf3e396"} Sep 10 11:13:03.247050 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.247028 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" event={"ID":"12e60f7f-5b9b-43d7-a097-a3d80233ad17","Type":"ContainerStarted","Data":"35f274715655cc5e6154e93f765932f194f173827ceebb753c5edb4d3fa3dc00"} Sep 10 11:13:03.248851 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.248826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" event={"ID":"f9576aa1f689d595f1c0703cd66e9d28","Type":"ContainerStarted","Data":"62b4cc98f53ce3ddc0d7b91459b5bf474e6f5a20e62a7976d3f7d2686af88629"} Sep 10 11:13:03.250561 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.250536 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mt6pz" event={"ID":"a1b5e602-0217-49c5-8c46-502120dff851","Type":"ContainerStarted","Data":"a58029bc0193d224b2b2cd7561591badb96d89b33b92ee3a3c49eaa6ac41a67a"} Sep 10 11:13:03.252105 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.252087 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52mbp" event={"ID":"bc0914eb-81b1-41b1-8553-83522073557c","Type":"ContainerStarted","Data":"f200fe37e3148950fccce3ac332b01fba6b5ccc57f21cf53d0e0a1b38a423f9e"} Sep 10 11:13:03.255059 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255043 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:13:03.255407 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255388 2560 generic.go:356] "Generic (PLEG): container finished" podID="811e7c53-3cb8-4df3-96d1-e1e823aa63ee" containerID="c05b82b73868d2307a5555041e2004a6c2b3da5e7bb6ede3709ad0c4c2f48e03" exitCode=1 Sep 10 11:13:03.255483 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255450 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"814e501d7bfc93560dc54c29385261382c88e7ea0d835dcf3d9391e913a785c3"} Sep 10 11:13:03.255545 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255513 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"749b1a4dab2bffbfb05ea4b83cc571019857c3ec29ba1b04f680e2f2a0b57094"} Sep 10 11:13:03.255588 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255552 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"7bec54d16fa5ef311ce8268d8594eb5699eb40b36b3002c47f57e6d5c5b99e9c"} Sep 10 11:13:03.255622 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255600 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"3182a5f8adb4c117785293f04c72d68e4c57d3410c3d964952558dcb761df266"} Sep 10 11:13:03.255665 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255629 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerDied","Data":"c05b82b73868d2307a5555041e2004a6c2b3da5e7bb6ede3709ad0c4c2f48e03"} Sep 10 11:13:03.255665 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.255661 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"afef41c88c3032129fc3f5a21d9a71b6068f4161abcda50a767e5abc684a25ba"} Sep 10 11:13:03.257019 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.257000 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4qq68" event={"ID":"70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b","Type":"ContainerStarted","Data":"feeef51fe3edc2341fb699bd1217d9207498526eca9f55a5257a047cc012046d"} Sep 10 11:13:03.258597 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.258578 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ppb4s" event={"ID":"aecc4dcd-2c08-453f-8202-1f571e23862a","Type":"ContainerStarted","Data":"616a64cf8a0423d05f23ec7e9471c1a36b4042371ea88aaf0f919e3a451a4795"} Sep 10 11:13:03.260126 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.260099 2560 generic.go:356] "Generic (PLEG): container finished" podID="eff84843-6d59-4eaf-ac58-1740f3ce992f" containerID="ae8ab7a35ca378a195c1cf2f52f7df2a148ecc355235623507b73d843f7810ad" exitCode=0 Sep 10 11:13:03.260205 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.260130 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerDied","Data":"ae8ab7a35ca378a195c1cf2f52f7df2a148ecc355235623507b73d843f7810ad"} Sep 10 11:13:03.268375 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.268336 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pqv7s" podStartSLOduration=3.21550152 podStartE2EDuration="21.268323049s" totalImagesPullingTime="18.052821529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:03.268014643 +0000 UTC m=+21.627070848" watchObservedRunningTime="2026-09-10 11:13:03.268323049 +0000 UTC m=+21.627379252" Sep 10 11:13:03.313969 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.313880 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ppb4s" podStartSLOduration=3.271433767 podStartE2EDuration="21.313859522s" totalImagesPullingTime="18.042425755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:03.312422833 +0000 UTC m=+21.671479034" watchObservedRunningTime="2026-09-10 11:13:03.313859522 +0000 UTC m=+21.672915730" Sep 10 11:13:03.334169 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.334119 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4qq68" podStartSLOduration=3.321082655 podStartE2EDuration="21.334103478s" totalImagesPullingTime="18.013020823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:03.334098085 +0000 UTC m=+21.693154290" watchObservedRunningTime="2026-09-10 11:13:03.334103478 +0000 UTC m=+21.693159684" Sep 10 11:13:03.353584 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.353538 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mt6pz" podStartSLOduration=3.350807896 podStartE2EDuration="21.353524403s" totalImagesPullingTime="18.002716507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:03.353276824 +0000 UTC m=+21.712333030" watchObservedRunningTime="2026-09-10 11:13:03.353524403 +0000 UTC m=+21.712580609" Sep 10 11:13:03.377013 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.376952 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-79.ec2.internal" podStartSLOduration=20.376932559 podStartE2EDuration="20.376932559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:03.376230241 +0000 UTC m=+21.735286443" watchObservedRunningTime="2026-09-10 11:13:03.376932559 +0000 UTC m=+21.735988768" Sep 10 11:13:03.404150 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.404098 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-52mbp" podStartSLOduration=2.771413246 podStartE2EDuration="21.40408233s" totalImagesPullingTime="18.632669084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:03.403383131 +0000 UTC m=+21.762439338" watchObservedRunningTime="2026-09-10 11:13:03.40408233 +0000 UTC m=+21.763138539" Sep 10 11:13:03.476333 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.476290 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pqv7s_b2b64fd3-a514-4a75-acec-78a1678d4044/dns-node-resolver/0.log" Sep 10 11:13:03.547728 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.547651 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:03.653091 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.653056 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:03.749752 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.749720 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:03.816029 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:03.816007 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 11:13:04.095176 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.095093 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:04.101174 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.101091 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T11:13:03.816026593Z","UUID":"c9fe52ba-446c-45d6-96f7-74745ae99717","Handler":null,"Name":"","Endpoint":""} Sep 10 11:13:04.102786 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.102742 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 11:13:04.102901 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.102798 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 11:13:04.166419 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.166379 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:04.166587 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:04.166489 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:04.166587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.166547 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:04.166669 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:04.166636 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:04.175520 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.175491 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:04.265452 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.265403 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" event={"ID":"12e60f7f-5b9b-43d7-a097-a3d80233ad17","Type":"ContainerStarted","Data":"50294492194f3482178625f5f31d1697bc9348aa0ccd8c5af12224e02592ce78"} Sep 10 11:13:04.267190 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.267150 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hf4jq" event={"ID":"a3997314-3e5b-49cf-b445-291aad983581","Type":"ContainerStarted","Data":"67b998ffb644c52972d38105d89f71b42d7bbdf93346891b43a488418a032a1b"} Sep 10 11:13:04.327217 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.327180 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:04.425186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.425159 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:13:04.425824 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.425810 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:13:04.449054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.448998 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hf4jq" podStartSLOduration=4.367934172 podStartE2EDuration="22.448982865s" totalImagesPullingTime="18.081048693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:04.290929407 +0000 UTC m=+22.649985613" watchObservedRunningTime="2026-09-10 11:13:04.448982865 +0000 UTC m=+22.808039070" Sep 10 11:13:04.664343 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:04.664313 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mt6pz_a1b5e602-0217-49c5-8c46-502120dff851/node-ca/0.log" Sep 10 11:13:05.166370 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:05.166271 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:05.166606 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:05.166424 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:05.274284 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:05.274252 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:13:05.274936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:05.274895 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"22c43bcb781798eda595236904db39e704a74f997e5f799921c656f9b004fa88"} Sep 10 11:13:05.278251 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:05.278222 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" event={"ID":"12e60f7f-5b9b-43d7-a097-a3d80233ad17","Type":"ContainerStarted","Data":"fcbcd24ab71feb1008b4f271adcee5548a65118b3803acd37f49558cfe3c0f1a"} Sep 10 11:13:05.300847 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:05.300795 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9kk57" podStartSLOduration=3.291633618 podStartE2EDuration="23.300774893s" totalImagesPullingTime="20.009141275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:05.300169942 +0000 UTC m=+23.659226147" watchObservedRunningTime="2026-09-10 11:13:05.300774893 +0000 UTC m=+23.659831100" Sep 10 11:13:06.167435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:06.167401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:06.167608 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:06.167562 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:06.167858 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:06.167401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:06.167979 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:06.167959 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:06.280926 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:06.280899 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 11:13:06.566712 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:06.566620 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:07.166621 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:07.166384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:07.166799 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:07.166736 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:08.166371 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:08.166335 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:08.166371 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:08.166375 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:08.166815 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:08.166485 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:08.166815 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:08.166600 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:09.076463 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.076273 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:09.166742 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.166709 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:09.167477 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:09.166818 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:09.173047 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.173019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:09.290830 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.290801 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:13:09.291217 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.291195 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"82622ff6b072b0648d24de6ca68292e1f236b1818aebe6dded9db4c5ffd00db7"} Sep 10 11:13:09.291454 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.291435 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:13:09.291687 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.291669 2560 scope.go:117] "RemoveContainer" containerID="c05b82b73868d2307a5555041e2004a6c2b3da5e7bb6ede3709ad0c4c2f48e03" Sep 10 11:13:09.293164 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.293136 2560 generic.go:356] "Generic (PLEG): container finished" podID="eff84843-6d59-4eaf-ac58-1740f3ce992f" containerID="ca94b9c60d50e00374c85cd160f542bad58507aa11aa3f9605fa34fb34373ffd" exitCode=0 Sep 10 11:13:09.293247 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.293182 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerDied","Data":"ca94b9c60d50e00374c85cd160f542bad58507aa11aa3f9605fa34fb34373ffd"} Sep 10 11:13:09.309039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.309014 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:13:09.395968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.395937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:09.479476 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.479441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:09.597235 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.597152 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:09.767805 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:09.767762 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:10.166010 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.165978 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:10.166181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.166015 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:10.166181 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:10.166110 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:10.166258 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:10.166236 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:10.302718 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.302684 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:13:10.303200 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.303164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" event={"ID":"811e7c53-3cb8-4df3-96d1-e1e823aa63ee","Type":"ContainerStarted","Data":"f8d769f5db178cd2ea323558d4b87329da657acc23cda5e1f1550224bd104f1d"} Sep 10 11:13:10.303427 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.303403 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:13:10.303500 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.303480 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:13:10.321355 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.321329 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:13:10.339630 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.339587 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" podStartSLOduration=10.254334663 podStartE2EDuration="28.339571981s" totalImagesPullingTime="18.085237318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:10.336822852 +0000 UTC m=+28.695879059" watchObservedRunningTime="2026-09-10 11:13:10.339571981 +0000 UTC m=+28.698628187" Sep 10 11:13:10.558025 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:10.557990 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:11.159035 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.158998 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:11.165926 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.165900 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:11.166030 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:11.165989 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:11.307464 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.307373 2560 generic.go:356] "Generic (PLEG): container finished" podID="eff84843-6d59-4eaf-ac58-1740f3ce992f" containerID="a44b85f7150cdf66aceef0aea751b0bbd14973cb440aa1626291f9b53e12905f" exitCode=0 Sep 10 11:13:11.307464 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.307424 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerDied","Data":"a44b85f7150cdf66aceef0aea751b0bbd14973cb440aa1626291f9b53e12905f"} Sep 10 11:13:11.359027 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.358994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:11.479184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.479155 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:13:11.479372 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.479257 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 11:13:11.479664 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.479647 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ppb4s" Sep 10 11:13:11.944579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:11.944540 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:12.016057 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:12.016020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:12.166707 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:12.166639 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:12.166850 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:12.166750 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:12.166850 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:12.166807 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:12.166955 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:12.166901 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:12.768537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:12.768502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:12.864723 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:12.864687 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:13.052463 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:13.052394 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:13.166498 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:13.166468 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:13.166653 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:13.166569 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:13.314677 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:13.314593 2560 generic.go:356] "Generic (PLEG): container finished" podID="eff84843-6d59-4eaf-ac58-1740f3ce992f" containerID="5f234a50654cf0d33c944a80587de97f747b4f81112f63ef84024697f00ee687" exitCode=0 Sep 10 11:13:13.314677 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:13.314643 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerDied","Data":"5f234a50654cf0d33c944a80587de97f747b4f81112f63ef84024697f00ee687"} Sep 10 11:13:13.344255 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:13.344218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:13.439195 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:13.439164 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:13.959934 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:13.959903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:14.166568 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:14.166530 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:14.166735 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:14.166665 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:14.166735 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:14.166718 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:14.166867 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:14.166826 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:14.247215 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:14.247125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:14.904141 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:14.904103 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.007704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.007672 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.152426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.152385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.166496 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.166472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:15.166634 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.166572 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:15.259195 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.259160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.361288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.361257 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.460654 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.460557 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.563492 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.563424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.752627 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.752543 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:15.895136 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.895096 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:15.895339 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.895156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:15.895339 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.895242 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:13:15.895339 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.895243 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 11:13:15.895339 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.895310 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs podName:f4f8581c-3527-40bb-89f6-b51b45a0d4af nodeName:}" failed. No retries permitted until 2026-09-10 11:13:47.895282356 +0000 UTC m=+66.254338540 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs") pod "network-metrics-daemon-r42wc" (UID: "f4f8581c-3527-40bb-89f6-b51b45a0d4af") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 11:13:15.895339 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.895324 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret podName:7f5e0a42-7fba-460a-b3d2-67a92ca2c94e nodeName:}" failed. No retries permitted until 2026-09-10 11:13:47.895317846 +0000 UTC m=+66.254374029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret") pod "global-pull-secret-syncer-dsmzj" (UID: "7f5e0a42-7fba-460a-b3d2-67a92ca2c94e") : object "kube-system"/"original-pull-secret" not registered Sep 10 11:13:15.996068 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:15.996029 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:15.996235 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.996176 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 11:13:15.996235 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.996198 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 11:13:15.996235 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.996212 2560 projected.go:196] Error preparing data for projected volume kube-api-access-sk5qj for pod openshift-network-diagnostics/network-check-target-9tdqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:13:15.996407 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:15.996276 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj podName:6ca7ad53-128f-4ce9-8dc7-ffa5482323c8 nodeName:}" failed. No retries permitted until 2026-09-10 11:13:47.996257824 +0000 UTC m=+66.355314031 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk5qj" (UniqueName: "kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj") pod "network-check-target-9tdqv" (UID: "6ca7ad53-128f-4ce9-8dc7-ffa5482323c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 11:13:16.166492 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:16.166468 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:16.166845 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:16.166472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:16.166845 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:16.166576 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:16.166845 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:16.166634 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:16.245700 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:16.245654 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:16.746918 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:16.746882 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:17.158890 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:17.158795 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:17.165719 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:17.165695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:17.165857 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:17.165802 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:17.584541 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:17.584488 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:17.681085 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:17.681050 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:17.791951 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:17.791918 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:18.166209 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:18.166163 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:18.166401 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:18.166163 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:18.166401 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:18.166314 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:18.166401 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:18.166368 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:18.963250 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:18.963206 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:19.149340 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:19.149285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:19.166427 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:19.166393 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:19.166542 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:19.166503 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:19.353267 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:19.353178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:19.549501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:19.549468 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:19.750183 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:19.750153 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:20.166386 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:20.166352 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:20.166762 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:20.166352 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:20.166762 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:20.166475 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:20.166762 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:20.166564 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:20.333074 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:20.333045 2560 generic.go:356] "Generic (PLEG): container finished" podID="eff84843-6d59-4eaf-ac58-1740f3ce992f" containerID="8e95b35589580e22883574ae7af07599228c438e829f299bbcefd8b2e500d7ab" exitCode=0 Sep 10 11:13:20.333227 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:20.333107 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerDied","Data":"8e95b35589580e22883574ae7af07599228c438e829f299bbcefd8b2e500d7ab"} Sep 10 11:13:21.166356 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:21.166324 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:21.166521 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:21.166422 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:21.338489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:21.338457 2560 generic.go:356] "Generic (PLEG): container finished" podID="eff84843-6d59-4eaf-ac58-1740f3ce992f" containerID="9d79b3ddec42f65cb0c791c4d5a55d9c7e8ecf06bb1afe0cf09165505bc7af96" exitCode=0 Sep 10 11:13:21.338643 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:21.338520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerDied","Data":"9d79b3ddec42f65cb0c791c4d5a55d9c7e8ecf06bb1afe0cf09165505bc7af96"} Sep 10 11:13:22.166876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:22.166669 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:22.167254 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:22.166729 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:22.167254 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:22.166946 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:22.167254 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:22.167020 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:22.343852 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:22.343818 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-244zd" event={"ID":"eff84843-6d59-4eaf-ac58-1740f3ce992f","Type":"ContainerStarted","Data":"25e2a8c99a433595309d97a8b05bd4abd412a87806d8202cd0207cc78e8fe8d3"} Sep 10 11:13:22.377933 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:22.377880 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-244zd" podStartSLOduration=9.311821989 podStartE2EDuration="40.377864824s" totalImagesPullingTime="31.066042835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:22.376859965 +0000 UTC m=+40.735916171" watchObservedRunningTime="2026-09-10 11:13:22.377864824 +0000 UTC m=+40.736921030" Sep 10 11:13:22.775421 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:22.775385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:22.955286 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:22.955254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.128015 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.127924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.166257 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.166233 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:23.166427 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:23.166351 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:23.233268 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.233241 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.421840 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.421801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.525741 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.525705 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.614778 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.614746 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.701087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.700986 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.801667 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.801615 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:23.904622 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:23.904594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:24.034985 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.034912 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:24.166248 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.166214 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:24.166408 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:24.166373 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:24.166469 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.166413 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:24.166520 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:24.166498 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:24.317671 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.317579 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r42wc"] Sep 10 11:13:24.323203 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.323170 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9tdqv"] Sep 10 11:13:24.323568 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.323550 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:24.323819 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:24.323795 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:24.326571 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.326545 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dsmzj"] Sep 10 11:13:24.348979 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.348946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:24.348979 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.348968 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:24.349146 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:24.349100 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:24.349229 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:24.349212 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:24.353775 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.353744 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:24.435720 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.435685 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:24.929462 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:24.929432 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:25.145544 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:25.145508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:25.354143 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:25.354063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:25.456359 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:25.456322 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:26.166258 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:26.166219 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:26.166258 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:26.166245 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:26.166498 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:26.166342 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:26.166498 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:26.166367 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:26.166498 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:26.166413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:26.166498 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:26.166466 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:26.562750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:26.562709 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:26.661404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:26.661372 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:27.738904 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:27.738868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:27.843791 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:27.843752 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:27.949083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:27.949048 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:28.161524 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.161442 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:28.166432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.166403 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:28.166574 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.166403 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:28.166574 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:28.166517 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsmzj" podUID="7f5e0a42-7fba-460a-b3d2-67a92ca2c94e" Sep 10 11:13:28.166689 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.166401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:28.166689 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:28.166589 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9tdqv" podUID="6ca7ad53-128f-4ce9-8dc7-ffa5482323c8" Sep 10 11:13:28.166689 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:28.166678 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r42wc" podUID="f4f8581c-3527-40bb-89f6-b51b45a0d4af" Sep 10 11:13:28.562993 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.562953 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:28.686561 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.686532 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:28.776185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.776146 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:28.863967 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.863934 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:28.957017 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.956940 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-79.ec2.internal" event="NodeReady" Sep 10 11:13:28.957172 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:28.957068 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 11:13:29.011010 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.010975 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9wdg7"] Sep 10 11:13:29.031289 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.031243 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jkmkp"] Sep 10 11:13:29.031475 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.031391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.034160 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.034140 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rpwp6\"" Sep 10 11:13:29.034393 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.034374 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 11:13:29.034477 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.034380 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 11:13:29.055522 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.055495 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9wdg7"] Sep 10 11:13:29.055672 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.055544 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jkmkp"] Sep 10 11:13:29.055672 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.055625 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.058332 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.058311 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 11:13:29.058453 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.058360 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 11:13:29.058624 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.058609 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 11:13:29.058696 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.058675 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d5csz\"" Sep 10 11:13:29.059098 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.059084 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 11:13:29.113418 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.113380 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f5dhc"] Sep 10 11:13:29.135439 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.135405 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5dhc"] Sep 10 11:13:29.135593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.135549 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.138171 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.138149 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 11:13:29.138400 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.138227 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 11:13:29.138795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.138778 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cs5mx\"" Sep 10 11:13:29.138932 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.138912 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 11:13:29.190092 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.190061 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:29.198315 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198273 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lfdq\" (UniqueName: \"kubernetes.io/projected/564e65e3-d691-42fe-afa5-2c86e533fc6f-kube-api-access-4lfdq\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.198450 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198322 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e718f029-cbb3-4784-bb09-bb828ec8ee3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.198450 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e718f029-cbb3-4784-bb09-bb828ec8ee3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.198450 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198432 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e718f029-cbb3-4784-bb09-bb828ec8ee3b-crio-socket\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.198554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/564e65e3-d691-42fe-afa5-2c86e533fc6f-config-volume\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.198554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564e65e3-d691-42fe-afa5-2c86e533fc6f-tmp-dir\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.198554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xch6z\" (UniqueName: \"kubernetes.io/projected/e718f029-cbb3-4784-bb09-bb828ec8ee3b-kube-api-access-xch6z\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.198554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e718f029-cbb3-4784-bb09-bb828ec8ee3b-data-volume\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.198554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.198552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/564e65e3-d691-42fe-afa5-2c86e533fc6f-metrics-tls\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.270349 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.270240 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:29.299800 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.299772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e718f029-cbb3-4784-bb09-bb828ec8ee3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.299913 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.299807 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e718f029-cbb3-4784-bb09-bb828ec8ee3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.299913 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.299884 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e718f029-cbb3-4784-bb09-bb828ec8ee3b-crio-socket\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.300004 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.299911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/564e65e3-d691-42fe-afa5-2c86e533fc6f-config-volume\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.300004 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.299934 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564e65e3-d691-42fe-afa5-2c86e533fc6f-tmp-dir\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.300004 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.299957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xch6z\" (UniqueName: \"kubernetes.io/projected/e718f029-cbb3-4784-bb09-bb828ec8ee3b-kube-api-access-xch6z\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.300004 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.299995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e718f029-cbb3-4784-bb09-bb828ec8ee3b-data-volume\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.300188 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b93c15f-256d-4f14-b104-cc20cadc496c-cert\") pod \"ingress-canary-f5dhc\" (UID: \"3b93c15f-256d-4f14-b104-cc20cadc496c\") " pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.300188 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300078 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/564e65e3-d691-42fe-afa5-2c86e533fc6f-metrics-tls\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.300188 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300089 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e718f029-cbb3-4784-bb09-bb828ec8ee3b-crio-socket\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.300188 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnpl8\" (UniqueName: \"kubernetes.io/projected/3b93c15f-256d-4f14-b104-cc20cadc496c-kube-api-access-jnpl8\") pod \"ingress-canary-f5dhc\" (UID: \"3b93c15f-256d-4f14-b104-cc20cadc496c\") " pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.300394 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lfdq\" (UniqueName: \"kubernetes.io/projected/564e65e3-d691-42fe-afa5-2c86e533fc6f-kube-api-access-4lfdq\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.300394 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300354 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564e65e3-d691-42fe-afa5-2c86e533fc6f-tmp-dir\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.300494 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300425 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e718f029-cbb3-4784-bb09-bb828ec8ee3b-data-volume\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.300662 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300638 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/564e65e3-d691-42fe-afa5-2c86e533fc6f-config-volume\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.300827 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.300807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e718f029-cbb3-4784-bb09-bb828ec8ee3b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.304677 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.304652 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/564e65e3-d691-42fe-afa5-2c86e533fc6f-metrics-tls\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.304776 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.304652 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e718f029-cbb3-4784-bb09-bb828ec8ee3b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.309629 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.309609 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xch6z\" (UniqueName: \"kubernetes.io/projected/e718f029-cbb3-4784-bb09-bb828ec8ee3b-kube-api-access-xch6z\") pod \"insights-runtime-extractor-jkmkp\" (UID: \"e718f029-cbb3-4784-bb09-bb828ec8ee3b\") " pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.309717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.309692 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lfdq\" (UniqueName: \"kubernetes.io/projected/564e65e3-d691-42fe-afa5-2c86e533fc6f-kube-api-access-4lfdq\") pod \"dns-default-9wdg7\" (UID: \"564e65e3-d691-42fe-afa5-2c86e533fc6f\") " pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.340654 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.340624 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:29.364231 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.364205 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jkmkp" Sep 10 11:13:29.400664 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.400631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b93c15f-256d-4f14-b104-cc20cadc496c-cert\") pod \"ingress-canary-f5dhc\" (UID: \"3b93c15f-256d-4f14-b104-cc20cadc496c\") " pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.400749 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.400677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnpl8\" (UniqueName: \"kubernetes.io/projected/3b93c15f-256d-4f14-b104-cc20cadc496c-kube-api-access-jnpl8\") pod \"ingress-canary-f5dhc\" (UID: \"3b93c15f-256d-4f14-b104-cc20cadc496c\") " pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.403201 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.403170 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b93c15f-256d-4f14-b104-cc20cadc496c-cert\") pod \"ingress-canary-f5dhc\" (UID: \"3b93c15f-256d-4f14-b104-cc20cadc496c\") " pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.413834 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.413803 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnpl8\" (UniqueName: \"kubernetes.io/projected/3b93c15f-256d-4f14-b104-cc20cadc496c-kube-api-access-jnpl8\") pod \"ingress-canary-f5dhc\" (UID: \"3b93c15f-256d-4f14-b104-cc20cadc496c\") " pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.445205 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.445146 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5dhc" Sep 10 11:13:29.675630 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.675423 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9wdg7"] Sep 10 11:13:29.676083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.676056 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jkmkp"] Sep 10 11:13:29.680345 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:29.680314 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564e65e3_d691_42fe_afa5_2c86e533fc6f.slice/crio-60be08e53790600c67a7d011b151e005e72f7d98a14d1dcfe4aa1bccc056f6ec WatchSource:0}: Error finding container 60be08e53790600c67a7d011b151e005e72f7d98a14d1dcfe4aa1bccc056f6ec: Status 404 returned error can't find the container with id 60be08e53790600c67a7d011b151e005e72f7d98a14d1dcfe4aa1bccc056f6ec Sep 10 11:13:29.680792 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:29.680760 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode718f029_cbb3_4784_bb09_bb828ec8ee3b.slice/crio-bc798b4dacc247b750ccc7b1235e809f5f4af1811ed9a9d3b8fbf03833e6a894 WatchSource:0}: Error finding container bc798b4dacc247b750ccc7b1235e809f5f4af1811ed9a9d3b8fbf03833e6a894: Status 404 returned error can't find the container with id bc798b4dacc247b750ccc7b1235e809f5f4af1811ed9a9d3b8fbf03833e6a894 Sep 10 11:13:29.748507 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:29.748426 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5dhc"] Sep 10 11:13:29.752943 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:29.752912 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b93c15f_256d_4f14_b104_cc20cadc496c.slice/crio-7e122f195c74f7f1ead35f4e12ba84192a9267bf0be10f09d112fd2be04da692 WatchSource:0}: Error finding container 7e122f195c74f7f1ead35f4e12ba84192a9267bf0be10f09d112fd2be04da692: Status 404 returned error can't find the container with id 7e122f195c74f7f1ead35f4e12ba84192a9267bf0be10f09d112fd2be04da692 Sep 10 11:13:30.169948 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.169920 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:30.170454 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.170010 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:30.170454 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.170133 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:30.172770 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.172750 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qdxwv\"" Sep 10 11:13:30.173706 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.173683 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 11:13:30.173706 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.173683 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 11:13:30.173884 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.173765 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 11:13:30.173884 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.173786 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 11:13:30.173884 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.173800 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t7cdh\"" Sep 10 11:13:30.363761 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.363716 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkmkp" event={"ID":"e718f029-cbb3-4784-bb09-bb828ec8ee3b","Type":"ContainerStarted","Data":"19b046e3d83fddb78771b2230c4e3c97f48b6a23fe15c3074ebeb641594179d2"} Sep 10 11:13:30.363906 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.363822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkmkp" event={"ID":"e718f029-cbb3-4784-bb09-bb828ec8ee3b","Type":"ContainerStarted","Data":"bc798b4dacc247b750ccc7b1235e809f5f4af1811ed9a9d3b8fbf03833e6a894"} Sep 10 11:13:30.364919 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.364890 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wdg7" event={"ID":"564e65e3-d691-42fe-afa5-2c86e533fc6f","Type":"ContainerStarted","Data":"60be08e53790600c67a7d011b151e005e72f7d98a14d1dcfe4aa1bccc056f6ec"} Sep 10 11:13:30.365955 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:30.365935 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5dhc" event={"ID":"3b93c15f-256d-4f14-b104-cc20cadc496c","Type":"ContainerStarted","Data":"7e122f195c74f7f1ead35f4e12ba84192a9267bf0be10f09d112fd2be04da692"} Sep 10 11:13:31.370883 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:31.370654 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkmkp" event={"ID":"e718f029-cbb3-4784-bb09-bb828ec8ee3b","Type":"ContainerStarted","Data":"02a29018053ca0ccc4d737ab55ae49d37e659716f312205bbd943c62a99f49f0"} Sep 10 11:13:32.401766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:32.401724 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:32.471429 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:32.471338 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:33.555039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.555017 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c45779ddc-qtqkm"] Sep 10 11:13:33.572340 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.572289 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.575283 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.575250 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c45779ddc-qtqkm"] Sep 10 11:13:33.576007 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.575879 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 11:13:33.576173 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.576130 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cmmkz\"" Sep 10 11:13:33.576257 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.576180 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 11:13:33.576331 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.576318 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 11:13:33.576413 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.576326 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 11:13:33.576968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.576661 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 11:13:33.576968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.576682 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 11:13:33.576968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.576733 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 11:13:33.638179 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.638144 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-config\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.638179 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.638186 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-service-ca\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.638426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.638221 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-oauth-serving-cert\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.638426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.638239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kc7w\" (UniqueName: \"kubernetes.io/projected/a4dda299-2891-4b1e-911b-97551f4e0cd4-kube-api-access-4kc7w\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.638426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.638334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-serving-cert\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.638426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.638396 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-oauth-config\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.739602 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.739570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-config\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.739728 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.739619 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-service-ca\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.739728 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.739665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-oauth-serving-cert\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.739728 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.739682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kc7w\" (UniqueName: \"kubernetes.io/projected/a4dda299-2891-4b1e-911b-97551f4e0cd4-kube-api-access-4kc7w\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.739879 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.739733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-serving-cert\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.739879 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.739759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-oauth-config\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.740486 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.740464 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-oauth-serving-cert\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.740658 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.740534 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-service-ca\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.741851 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.741827 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-config\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.745768 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.745627 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-oauth-config\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.745878 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.745628 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-serving-cert\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.750419 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.750396 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kc7w\" (UniqueName: \"kubernetes.io/projected/a4dda299-2891-4b1e-911b-97551f4e0cd4-kube-api-access-4kc7w\") pod \"console-5c45779ddc-qtqkm\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.887083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.887041 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:33.988778 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:33.988747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:34.026683 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.026650 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8rgk6"] Sep 10 11:13:34.050446 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.050417 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8rgk6"] Sep 10 11:13:34.050592 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.050523 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.053224 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.053201 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 11:13:34.053464 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.053442 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 11:13:34.053643 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.053631 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 10 11:13:34.053715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.053654 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lzpvn\"" Sep 10 11:13:34.053715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.053663 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 10 11:13:34.053906 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.053893 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 11:13:34.100501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.100469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:34.142818 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.142782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.142818 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.142823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.143036 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.142847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.143036 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.142927 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnq47\" (UniqueName: \"kubernetes.io/projected/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-kube-api-access-pnq47\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.173293 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.173264 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c45779ddc-qtqkm"] Sep 10 11:13:34.177115 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:34.177091 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dda299_2891_4b1e_911b_97551f4e0cd4.slice/crio-fa49abe9f0b2129a8c56653b3aea1a7cb08c56be9c4f730b349b2d01bf019935 WatchSource:0}: Error finding container fa49abe9f0b2129a8c56653b3aea1a7cb08c56be9c4f730b349b2d01bf019935: Status 404 returned error can't find the container with id fa49abe9f0b2129a8c56653b3aea1a7cb08c56be9c4f730b349b2d01bf019935 Sep 10 11:13:34.243352 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.243252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnq47\" (UniqueName: \"kubernetes.io/projected/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-kube-api-access-pnq47\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.243352 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.243327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.243575 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.243365 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.243575 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.243396 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.243575 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:34.243482 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 10 11:13:34.243575 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:34.243561 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-tls podName:bf8f48fd-100a-4ce0-ba11-03c9cbe056ba nodeName:}" failed. No retries permitted until 2026-09-10 11:13:34.743538852 +0000 UTC m=+53.102595059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-8rgk6" (UID: "bf8f48fd-100a-4ce0-ba11-03c9cbe056ba") : secret "prometheus-operator-tls" not found Sep 10 11:13:34.244059 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.244039 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.246463 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.246444 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.252469 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.252445 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnq47\" (UniqueName: \"kubernetes.io/projected/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-kube-api-access-pnq47\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.381213 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.381181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkmkp" event={"ID":"e718f029-cbb3-4784-bb09-bb828ec8ee3b","Type":"ContainerStarted","Data":"b6cb959706769812f27cebee2ae4d5926a7f661e461b312edebcbd1cbab38f36"} Sep 10 11:13:34.382536 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.382504 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c45779ddc-qtqkm" event={"ID":"a4dda299-2891-4b1e-911b-97551f4e0cd4","Type":"ContainerStarted","Data":"fa49abe9f0b2129a8c56653b3aea1a7cb08c56be9c4f730b349b2d01bf019935"} Sep 10 11:13:34.384349 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.384326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wdg7" event={"ID":"564e65e3-d691-42fe-afa5-2c86e533fc6f","Type":"ContainerStarted","Data":"42d94e4ef3ca75ff2efec7308d2a81c52783eecc337fe43e2b493032903d7886"} Sep 10 11:13:34.384461 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.384372 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9wdg7" event={"ID":"564e65e3-d691-42fe-afa5-2c86e533fc6f","Type":"ContainerStarted","Data":"ca4c9d3330800977a51f2cd6b2f4bb575a2ff4ec2d2af7bda037ab5e316529c2"} Sep 10 11:13:34.384461 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.384401 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:34.385761 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.385740 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5dhc" event={"ID":"3b93c15f-256d-4f14-b104-cc20cadc496c","Type":"ContainerStarted","Data":"f868ea922d9414e3039034ae7a8f1de01d11dfb4963dff38fd5ad82c96b587b9"} Sep 10 11:13:34.403290 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.403054 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jkmkp" podStartSLOduration=2.650580397 podStartE2EDuration="6.403034414s" totalImagesPullingTime="3.752454017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:13:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:34.402418132 +0000 UTC m=+52.761474339" watchObservedRunningTime="2026-09-10 11:13:34.403034414 +0000 UTC m=+52.762090619" Sep 10 11:13:34.423066 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.423008 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f5dhc" podStartSLOduration=1.733965429 podStartE2EDuration="5.422988212s" totalImagesPullingTime="3.689022783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:34.421516586 +0000 UTC m=+52.780572792" watchObservedRunningTime="2026-09-10 11:13:34.422988212 +0000 UTC m=+52.782044420" Sep 10 11:13:34.446435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.446377 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9wdg7" podStartSLOduration=2.684763957 podStartE2EDuration="6.446358822s" totalImagesPullingTime="3.761594865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:34.444719402 +0000 UTC m=+52.803775610" watchObservedRunningTime="2026-09-10 11:13:34.446358822 +0000 UTC m=+52.805415027" Sep 10 11:13:34.460995 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.460965 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:34.747537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.747502 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.750130 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.750103 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf8f48fd-100a-4ce0-ba11-03c9cbe056ba-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-8rgk6\" (UID: \"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:34.959268 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:34.959228 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" Sep 10 11:13:35.229490 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:35.229266 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-8rgk6"] Sep 10 11:13:35.233400 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:35.233372 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8f48fd_100a_4ce0_ba11_03c9cbe056ba.slice/crio-9cb65d96188037f45d8aaadf4566d999c284bb233c3c00040d80682682536e24 WatchSource:0}: Error finding container 9cb65d96188037f45d8aaadf4566d999c284bb233c3c00040d80682682536e24: Status 404 returned error can't find the container with id 9cb65d96188037f45d8aaadf4566d999c284bb233c3c00040d80682682536e24 Sep 10 11:13:35.390697 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:35.390605 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" event={"ID":"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba","Type":"ContainerStarted","Data":"9cb65d96188037f45d8aaadf4566d999c284bb233c3c00040d80682682536e24"} Sep 10 11:13:36.227956 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:36.227914 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:36.362804 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:36.362772 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:37.919520 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:37.919474 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:38.394612 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.394505 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:38.401318 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.401273 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c45779ddc-qtqkm" event={"ID":"a4dda299-2891-4b1e-911b-97551f4e0cd4","Type":"ContainerStarted","Data":"88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1"} Sep 10 11:13:38.403091 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.403068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" event={"ID":"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba","Type":"ContainerStarted","Data":"17d9bfbbab3745c686ac574c73fa4a019819bdfc110a6d82e1935200970813a7"} Sep 10 11:13:38.403195 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.403111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" event={"ID":"bf8f48fd-100a-4ce0-ba11-03c9cbe056ba","Type":"ContainerStarted","Data":"4efedfc1b509176c354309cc085075103c14a47fe4c195efced137a61e979f9b"} Sep 10 11:13:38.425087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.425031 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c45779ddc-qtqkm" podStartSLOduration=2.014687602 podStartE2EDuration="5.425014613s" totalImagesPullingTime="3.410327011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:38.423574955 +0000 UTC m=+56.782631161" watchObservedRunningTime="2026-09-10 11:13:38.425014613 +0000 UTC m=+56.784070818" Sep 10 11:13:38.446531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.446477 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-8rgk6" podStartSLOduration=2.097996228 podStartE2EDuration="4.446461159s" totalImagesPullingTime="2.348464931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:38.445995528 +0000 UTC m=+56.805051733" watchObservedRunningTime="2026-09-10 11:13:38.446461159 +0000 UTC m=+56.805517359" Sep 10 11:13:38.482125 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.482099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:38.584494 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:38.584459 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:40.170041 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.169995 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:40.346954 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.346920 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:40.433493 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.433414 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-459rp"] Sep 10 11:13:40.461711 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.461667 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:40.467334 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.467287 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-459rp"] Sep 10 11:13:40.467495 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.467412 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.468743 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.468720 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5xtzc"] Sep 10 11:13:40.470458 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.470438 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 11:13:40.470555 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.470530 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-q2lx7\"" Sep 10 11:13:40.470880 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.470865 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 11:13:40.481303 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.481283 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-x84rv"] Sep 10 11:13:40.481438 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.481422 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.488989 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.488970 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 11:13:40.489949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.489933 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 11:13:40.490039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.490025 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x7xds\"" Sep 10 11:13:40.490145 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.490132 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 11:13:40.491329 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m56wl\" (UniqueName: \"kubernetes.io/projected/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-kube-api-access-m56wl\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491419 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.491419 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-metrics-client-ca\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491494 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491441 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q87nl\" (UniqueName: \"kubernetes.io/projected/16196610-c43e-4d83-a1e5-06d39a0e3bfc-kube-api-access-q87nl\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.491494 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491463 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-textfile\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491494 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491478 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-tls\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491610 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491501 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-wtmp\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491610 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491610 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16196610-c43e-4d83-a1e5-06d39a0e3bfc-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.491717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-sys\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-root\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.491874 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.491758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.499281 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.499256 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-x84rv"] Sep 10 11:13:40.499409 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.499365 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.501808 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.501790 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 11:13:40.501943 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.501926 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-djdsn\"" Sep 10 11:13:40.502193 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.502165 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 11:13:40.502365 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.502349 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 11:13:40.592630 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acbc0d41-f803-4bde-9b12-3042136b601e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.592819 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87nl\" (UniqueName: \"kubernetes.io/projected/16196610-c43e-4d83-a1e5-06d39a0e3bfc-kube-api-access-q87nl\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.592819 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-textfile\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.592819 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592751 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-tls\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.592819 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-wtmp\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.592819 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16196610-c43e-4d83-a1e5-06d39a0e3bfc-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acbc0d41-f803-4bde-9b12-3042136b601e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-sys\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592948 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592960 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-wtmp\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.592974 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:40.592996 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593010 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-sys\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593069 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:40.593066 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-tls podName:7d8d509c-426a-4ac0-a3dd-90c2756bff1f nodeName:}" failed. No retries permitted until 2026-09-10 11:13:41.093048023 +0000 UTC m=+59.452104228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-tls") pod "node-exporter-5xtzc" (UID: "7d8d509c-426a-4ac0-a3dd-90c2756bff1f") : secret "node-exporter-tls" not found Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593000 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593153 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-root\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593171 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-textfile\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593229 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m56wl\" (UniqueName: \"kubernetes.io/projected/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-kube-api-access-m56wl\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:40.593256 2560 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593267 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:40.593329 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-tls podName:16196610-c43e-4d83-a1e5-06d39a0e3bfc nodeName:}" failed. No retries permitted until 2026-09-10 11:13:41.093314573 +0000 UTC m=+59.452370771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-459rp" (UID: "16196610-c43e-4d83-a1e5-06d39a0e3bfc") : secret "openshift-state-metrics-tls" not found Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593255 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-root\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-metrics-client-ca\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593405 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntkj5\" (UniqueName: \"kubernetes.io/projected/acbc0d41-f803-4bde-9b12-3042136b601e-kube-api-access-ntkj5\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.593550 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593536 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.594126 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593604 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16196610-c43e-4d83-a1e5-06d39a0e3bfc-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.594126 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.593949 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-metrics-client-ca\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.595929 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.595902 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.596029 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.596019 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.603630 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.603609 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87nl\" (UniqueName: \"kubernetes.io/projected/16196610-c43e-4d83-a1e5-06d39a0e3bfc-kube-api-access-q87nl\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:40.603846 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.603825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m56wl\" (UniqueName: \"kubernetes.io/projected/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-kube-api-access-m56wl\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:40.693841 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.693755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acbc0d41-f803-4bde-9b12-3042136b601e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.693841 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.693797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.693841 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.693830 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.694111 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.693893 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.694111 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.693931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntkj5\" (UniqueName: \"kubernetes.io/projected/acbc0d41-f803-4bde-9b12-3042136b601e-kube-api-access-ntkj5\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.694111 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.693971 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acbc0d41-f803-4bde-9b12-3042136b601e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.694111 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:40.694015 2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 10 11:13:40.694111 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:40.694093 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-tls podName:acbc0d41-f803-4bde-9b12-3042136b601e nodeName:}" failed. No retries permitted until 2026-09-10 11:13:41.19407384 +0000 UTC m=+59.553130046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-x84rv" (UID: "acbc0d41-f803-4bde-9b12-3042136b601e") : secret "kube-state-metrics-tls" not found Sep 10 11:13:40.694375 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.694265 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acbc0d41-f803-4bde-9b12-3042136b601e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.694664 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.694640 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.694785 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.694641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acbc0d41-f803-4bde-9b12-3042136b601e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.694853 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.694838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:40.705719 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.705693 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntkj5\" (UniqueName: \"kubernetes.io/projected/acbc0d41-f803-4bde-9b12-3042136b601e-kube-api-access-ntkj5\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.708430 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.708398 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:40.800474 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.800447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:40.895890 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:40.895855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:41.009441 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.009347 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:41.096694 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.096660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:41.097719 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.097698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:41.097789 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.097778 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-tls\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:41.099963 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.099946 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7d8d509c-426a-4ac0-a3dd-90c2756bff1f-node-exporter-tls\") pod \"node-exporter-5xtzc\" (UID: \"7d8d509c-426a-4ac0-a3dd-90c2756bff1f\") " pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:41.100029 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.100002 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/16196610-c43e-4d83-a1e5-06d39a0e3bfc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-459rp\" (UID: \"16196610-c43e-4d83-a1e5-06d39a0e3bfc\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:41.198795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.198753 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:41.201278 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.201252 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acbc0d41-f803-4bde-9b12-3042136b601e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-x84rv\" (UID: \"acbc0d41-f803-4bde-9b12-3042136b601e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:41.280229 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.280151 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:41.354979 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.354947 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65f7759f67-sn6n4"] Sep 10 11:13:41.357983 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.357965 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.366721 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.366699 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 11:13:41.373821 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.373798 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f7759f67-sn6n4"] Sep 10 11:13:41.376943 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.376919 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" Sep 10 11:13:41.389427 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.389397 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5xtzc" Sep 10 11:13:41.399728 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:41.399703 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8d509c_426a_4ac0_a3dd_90c2756bff1f.slice/crio-88809c1a2b1ab59751b74f82c06b25a3abe11e332b4cdb2daaa8e544149226d7 WatchSource:0}: Error finding container 88809c1a2b1ab59751b74f82c06b25a3abe11e332b4cdb2daaa8e544149226d7: Status 404 returned error can't find the container with id 88809c1a2b1ab59751b74f82c06b25a3abe11e332b4cdb2daaa8e544149226d7 Sep 10 11:13:41.400481 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.400458 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-trusted-ca-bundle\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.400595 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.400517 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-console-config\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.400595 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.400577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzbjp\" (UniqueName: \"kubernetes.io/projected/628440b1-3553-4e6a-a46c-25482c0bac94-kube-api-access-xzbjp\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.400705 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.400646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-service-ca\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.400705 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.400694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-oauth-config\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.400789 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.400721 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-serving-cert\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.400923 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.400907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-oauth-serving-cert\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.408235 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.408208 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" Sep 10 11:13:41.418088 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.418059 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5xtzc" event={"ID":"7d8d509c-426a-4ac0-a3dd-90c2756bff1f","Type":"ContainerStarted","Data":"88809c1a2b1ab59751b74f82c06b25a3abe11e332b4cdb2daaa8e544149226d7"} Sep 10 11:13:41.481626 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.481570 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:41.501843 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.501808 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-service-ca\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.501949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.501867 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-oauth-config\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.501949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.501901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-serving-cert\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.501949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.501931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-oauth-serving-cert\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.502113 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.501966 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-trusted-ca-bundle\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.502113 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.501997 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-console-config\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.502113 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.502075 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzbjp\" (UniqueName: \"kubernetes.io/projected/628440b1-3553-4e6a-a46c-25482c0bac94-kube-api-access-xzbjp\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.502651 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.502603 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-service-ca\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.502753 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.502678 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-oauth-serving-cert\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.502840 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.502810 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-console-config\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.503128 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.503111 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-trusted-ca-bundle\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.504239 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.504219 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-oauth-config\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.504484 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.504468 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-serving-cert\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.512215 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.512195 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzbjp\" (UniqueName: \"kubernetes.io/projected/628440b1-3553-4e6a-a46c-25482c0bac94-kube-api-access-xzbjp\") pod \"console-65f7759f67-sn6n4\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.622774 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.622664 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:41.628771 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.628744 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:13:41.631823 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.631803 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.634809 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.634787 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 11:13:41.634938 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.634918 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5zss5\"" Sep 10 11:13:41.635348 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.635310 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 11:13:41.635482 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.635348 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 11:13:41.635482 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.635366 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 11:13:41.636470 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.636437 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 11:13:41.636547 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.636448 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 11:13:41.636894 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.636870 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 11:13:41.636894 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.636881 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 11:13:41.637552 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.637534 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 11:13:41.658538 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.657142 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:13:41.664636 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.664608 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-459rp"] Sep 10 11:13:41.667925 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.667833 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:41.669969 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:41.669924 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16196610_c43e_4d83_a1e5_06d39a0e3bfc.slice/crio-a27badbb23f1c668f49c07a29a40a0a69fe9c75a0aeae0b80738c839cf0e7fa3 WatchSource:0}: Error finding container a27badbb23f1c668f49c07a29a40a0a69fe9c75a0aeae0b80738c839cf0e7fa3: Status 404 returned error can't find the container with id a27badbb23f1c668f49c07a29a40a0a69fe9c75a0aeae0b80738c839cf0e7fa3 Sep 10 11:13:41.703116 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703084 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703269 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703129 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703269 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703148 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdqdk\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-kube-api-access-jdqdk\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703269 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703167 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-web-config\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703269 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703269 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703524 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703524 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703405 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-out\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703524 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703445 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-volume\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703524 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703472 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703524 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703496 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703693 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703540 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.703693 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.703562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.710350 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.710321 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-x84rv"] Sep 10 11:13:41.721369 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.721295 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:41.725971 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:41.725931 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacbc0d41_f803_4bde_9b12_3042136b601e.slice/crio-42a8abecabfe8cf08adc389826c367fafdfeae1ab1e8e9ca74672a206a20aba7 WatchSource:0}: Error finding container 42a8abecabfe8cf08adc389826c367fafdfeae1ab1e8e9ca74672a206a20aba7: Status 404 returned error can't find the container with id 42a8abecabfe8cf08adc389826c367fafdfeae1ab1e8e9ca74672a206a20aba7 Sep 10 11:13:41.804357 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdqdk\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-kube-api-access-jdqdk\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804395 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-web-config\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804472 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804511 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-out\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804873 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-volume\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.804873 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:41.804588 2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 10 11:13:41.804873 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:41.804662 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls podName:ccf8c940-fc81-4ccc-a4e0-a29d8dac0934 nodeName:}" failed. No retries permitted until 2026-09-10 11:13:42.304643786 +0000 UTC m=+60.663699992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934") : secret "alertmanager-main-tls" not found Sep 10 11:13:41.805028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804890 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.805028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.805028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.804977 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.805172 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.805125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.805233 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.805214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.805687 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.805395 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.806094 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.806067 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.806590 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.806564 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.808441 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.808003 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-volume\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.808441 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.808058 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.808441 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.808391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.808680 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.808460 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-web-config\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.808680 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.808573 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.808680 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.808584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.808680 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.808592 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-out\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.809345 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.809325 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.814165 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.814144 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdqdk\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-kube-api-access-jdqdk\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:41.949051 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:41.949018 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f7759f67-sn6n4"] Sep 10 11:13:41.952797 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:41.952771 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628440b1_3553_4e6a_a46c_25482c0bac94.slice/crio-268e517565287b910d8c6a2239a53f052efd66fe03c517624cfc8f1bfb5372a7 WatchSource:0}: Error finding container 268e517565287b910d8c6a2239a53f052efd66fe03c517624cfc8f1bfb5372a7: Status 404 returned error can't find the container with id 268e517565287b910d8c6a2239a53f052efd66fe03c517624cfc8f1bfb5372a7 Sep 10 11:13:42.147562 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.147525 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:42.310322 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.310206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:42.312627 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.312602 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:42.320664 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.320642 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v4578" Sep 10 11:13:42.424362 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.424320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" event={"ID":"acbc0d41-f803-4bde-9b12-3042136b601e","Type":"ContainerStarted","Data":"42a8abecabfe8cf08adc389826c367fafdfeae1ab1e8e9ca74672a206a20aba7"} Sep 10 11:13:42.426461 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.426425 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" event={"ID":"16196610-c43e-4d83-a1e5-06d39a0e3bfc","Type":"ContainerStarted","Data":"83bb5c22294317a789a4913e962193fe7ab9e1c7622bc1160070cbe0b033ef79"} Sep 10 11:13:42.426596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.426494 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" event={"ID":"16196610-c43e-4d83-a1e5-06d39a0e3bfc","Type":"ContainerStarted","Data":"cc8dc536364c2a28ee3fe0eb162536180207fff25c3d0ae235cca0f8657c1a58"} Sep 10 11:13:42.426596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.426527 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" event={"ID":"16196610-c43e-4d83-a1e5-06d39a0e3bfc","Type":"ContainerStarted","Data":"a27badbb23f1c668f49c07a29a40a0a69fe9c75a0aeae0b80738c839cf0e7fa3"} Sep 10 11:13:42.428254 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.428225 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7759f67-sn6n4" event={"ID":"628440b1-3553-4e6a-a46c-25482c0bac94","Type":"ContainerStarted","Data":"0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be"} Sep 10 11:13:42.428399 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.428270 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7759f67-sn6n4" event={"ID":"628440b1-3553-4e6a-a46c-25482c0bac94","Type":"ContainerStarted","Data":"268e517565287b910d8c6a2239a53f052efd66fe03c517624cfc8f1bfb5372a7"} Sep 10 11:13:42.456186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.456125 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65f7759f67-sn6n4" podStartSLOduration=1.4561060860000001 podStartE2EDuration="1.456106086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:42.454125858 +0000 UTC m=+60.813182063" watchObservedRunningTime="2026-09-10 11:13:42.456106086 +0000 UTC m=+60.815162292" Sep 10 11:13:42.500061 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.500024 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:42.547098 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.547064 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5zss5\"" Sep 10 11:13:42.553568 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.553541 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:13:42.577759 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.577724 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-74f774c495-5crkk"] Sep 10 11:13:42.580263 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.580229 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.583280 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.583255 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 11:13:42.583808 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.583782 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dpsct\"" Sep 10 11:13:42.584000 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.583980 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 11:13:42.584096 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.584065 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 11:13:42.584338 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.584295 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 11:13:42.584421 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.584402 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3r5t3dl1im947\"" Sep 10 11:13:42.584498 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.584483 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 11:13:42.602064 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.602032 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74f774c495-5crkk"] Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614164 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb05e7bf-65d9-4b36-a486-34294952a59c-metrics-client-ca\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614232 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614283 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgcxc\" (UniqueName: \"kubernetes.io/projected/eb05e7bf-65d9-4b36-a486-34294952a59c-kube-api-access-sgcxc\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614337 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614372 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-tls\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614409 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614480 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-grpc-tls\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.616674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.614507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715004 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.714960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715228 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.715036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgcxc\" (UniqueName: \"kubernetes.io/projected/eb05e7bf-65d9-4b36-a486-34294952a59c-kube-api-access-sgcxc\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715228 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.715071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715228 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.715094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-tls\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715440 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.715295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715440 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.715411 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-grpc-tls\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.715445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.715584 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.715527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb05e7bf-65d9-4b36-a486-34294952a59c-metrics-client-ca\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.716329 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.716262 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb05e7bf-65d9-4b36-a486-34294952a59c-metrics-client-ca\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.718203 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.718154 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.718436 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.718411 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.718529 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.718489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-tls\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.718592 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.718554 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.718694 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.718672 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.718995 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.718970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eb05e7bf-65d9-4b36-a486-34294952a59c-secret-grpc-tls\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.729847 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.729820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgcxc\" (UniqueName: \"kubernetes.io/projected/eb05e7bf-65d9-4b36-a486-34294952a59c-kube-api-access-sgcxc\") pod \"thanos-querier-74f774c495-5crkk\" (UID: \"eb05e7bf-65d9-4b36-a486-34294952a59c\") " pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:42.850841 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.850743 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:13:42.864413 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:42.864382 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf8c940_fc81_4ccc_a4e0_a29d8dac0934.slice/crio-21c603977e05ec0b964fcd0dc4f7e75c07f0d387643fe6a5687743d860f3bea2 WatchSource:0}: Error finding container 21c603977e05ec0b964fcd0dc4f7e75c07f0d387643fe6a5687743d860f3bea2: Status 404 returned error can't find the container with id 21c603977e05ec0b964fcd0dc4f7e75c07f0d387643fe6a5687743d860f3bea2 Sep 10 11:13:42.891291 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:42.891256 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:43.087602 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.087515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:43.189271 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.189127 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74f774c495-5crkk"] Sep 10 11:13:43.190005 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.189979 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:43.309865 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:43.309829 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb05e7bf_65d9_4b36_a486_34294952a59c.slice/crio-66567b58ccec72d80517b4beb911e9640868ddd91dec3aff89e29aa430eed5ad WatchSource:0}: Error finding container 66567b58ccec72d80517b4beb911e9640868ddd91dec3aff89e29aa430eed5ad: Status 404 returned error can't find the container with id 66567b58ccec72d80517b4beb911e9640868ddd91dec3aff89e29aa430eed5ad Sep 10 11:13:43.434763 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.434725 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerStarted","Data":"21c603977e05ec0b964fcd0dc4f7e75c07f0d387643fe6a5687743d860f3bea2"} Sep 10 11:13:43.440734 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.440663 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" event={"ID":"16196610-c43e-4d83-a1e5-06d39a0e3bfc","Type":"ContainerStarted","Data":"3b0c241d6ff7fdd345dd87168f0105271a3b74bc16016bba4e3ed1543aefa2a1"} Sep 10 11:13:43.442286 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.442258 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" event={"ID":"eb05e7bf-65d9-4b36-a486-34294952a59c","Type":"ContainerStarted","Data":"66567b58ccec72d80517b4beb911e9640868ddd91dec3aff89e29aa430eed5ad"} Sep 10 11:13:43.474328 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.473390 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-459rp" podStartSLOduration=2.462698426 podStartE2EDuration="3.473371576s" totalImagesPullingTime="1.01067315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:43.472581138 +0000 UTC m=+61.831637345" watchObservedRunningTime="2026-09-10 11:13:43.473371576 +0000 UTC m=+61.832427783" Sep 10 11:13:43.484658 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.484613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:43.583485 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.583448 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:43.762731 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.762700 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:43.859498 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.859468 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:43.887186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.887135 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:43.887331 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.887236 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:43.893394 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:43.893326 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:44.393177 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.393145 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9wdg7" Sep 10 11:13:44.450649 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.450612 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" event={"ID":"acbc0d41-f803-4bde-9b12-3042136b601e","Type":"ContainerStarted","Data":"bd21fcf6cd9f3e08371fbce0c30bc3bad28408b44f77579780d4ef9c3159f41e"} Sep 10 11:13:44.451054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.450678 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" event={"ID":"acbc0d41-f803-4bde-9b12-3042136b601e","Type":"ContainerStarted","Data":"08612adf19fe27c72cfa908bbbe7a5d25b3b5ffeb4e464dc3ab670024fca84da"} Sep 10 11:13:44.451054 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.450699 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" event={"ID":"acbc0d41-f803-4bde-9b12-3042136b601e","Type":"ContainerStarted","Data":"c53513aa26f4096d1b4f86e06336280cc92bcaa67729f521fe0108160e07441a"} Sep 10 11:13:44.453578 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.453551 2560 generic.go:356] "Generic (PLEG): container finished" podID="7d8d509c-426a-4ac0-a3dd-90c2756bff1f" containerID="e08a61ba8598970e97e8782fe1a11de7750914fb58daca02e2db6e6b62f0bef3" exitCode=0 Sep 10 11:13:44.453706 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.453641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5xtzc" event={"ID":"7d8d509c-426a-4ac0-a3dd-90c2756bff1f","Type":"ContainerDied","Data":"e08a61ba8598970e97e8782fe1a11de7750914fb58daca02e2db6e6b62f0bef3"} Sep 10 11:13:44.459935 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.459891 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:13:44.477276 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.477183 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-x84rv" podStartSLOduration=2.556677337 podStartE2EDuration="4.477163469s" totalImagesPullingTime="1.920486132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:44.475658196 +0000 UTC m=+62.834714403" watchObservedRunningTime="2026-09-10 11:13:44.477163469 +0000 UTC m=+62.836219674" Sep 10 11:13:44.534433 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.534396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:44.633559 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.633528 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:44.849237 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.849188 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-ff66b8fb-9xtkv"] Sep 10 11:13:44.852405 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.852376 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:44.856614 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.856582 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 10 11:13:44.857257 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.857233 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dt71sp69au0s5\"" Sep 10 11:13:44.857681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.857659 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 10 11:13:44.857766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.857731 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 11:13:44.857819 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.857779 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 10 11:13:44.858018 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.857999 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5fbgq\"" Sep 10 11:13:44.863113 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.863091 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-ff66b8fb-9xtkv"] Sep 10 11:13:44.940419 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.940378 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b3bc84d1-9c77-4f59-9674-2439981a24af-audit-log\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:44.940575 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.940447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b3bc84d1-9c77-4f59-9674-2439981a24af-metrics-server-audit-profiles\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:44.940575 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.940481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr96v\" (UniqueName: \"kubernetes.io/projected/b3bc84d1-9c77-4f59-9674-2439981a24af-kube-api-access-mr96v\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:44.940575 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.940518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-secret-metrics-server-client-certs\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:44.940711 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.940583 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3bc84d1-9c77-4f59-9674-2439981a24af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:44.940742 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.940712 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-client-ca-bundle\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:44.940781 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:44.940745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-secret-metrics-server-tls\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041344 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041291 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-client-ca-bundle\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-secret-metrics-server-tls\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b3bc84d1-9c77-4f59-9674-2439981a24af-audit-log\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b3bc84d1-9c77-4f59-9674-2439981a24af-metrics-server-audit-profiles\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041449 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr96v\" (UniqueName: \"kubernetes.io/projected/b3bc84d1-9c77-4f59-9674-2439981a24af-kube-api-access-mr96v\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-secret-metrics-server-client-certs\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041531 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3bc84d1-9c77-4f59-9674-2439981a24af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.041823 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.041807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b3bc84d1-9c77-4f59-9674-2439981a24af-audit-log\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.042507 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.042475 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3bc84d1-9c77-4f59-9674-2439981a24af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.042700 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.042665 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b3bc84d1-9c77-4f59-9674-2439981a24af-metrics-server-audit-profiles\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.044338 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.044312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-client-ca-bundle\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.044456 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.044345 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-secret-metrics-server-tls\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.044555 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.044532 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b3bc84d1-9c77-4f59-9674-2439981a24af-secret-metrics-server-client-certs\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.051044 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.051022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr96v\" (UniqueName: \"kubernetes.io/projected/b3bc84d1-9c77-4f59-9674-2439981a24af-kube-api-access-mr96v\") pod \"metrics-server-ff66b8fb-9xtkv\" (UID: \"b3bc84d1-9c77-4f59-9674-2439981a24af\") " pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.164039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.163962 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:13:45.171960 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.171925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:45.199619 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.199587 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69"] Sep 10 11:13:45.204443 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.204395 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.207144 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.207115 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2cwqq\"" Sep 10 11:13:45.207460 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.207443 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 11:13:45.216766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.216740 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69"] Sep 10 11:13:45.243733 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.243565 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21d22bca-990d-44c3-967d-1989bf231b5f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-72m69\" (UID: \"21d22bca-990d-44c3-967d-1989bf231b5f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.243733 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.243609 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k25ll\" (UniqueName: \"kubernetes.io/projected/21d22bca-990d-44c3-967d-1989bf231b5f-kube-api-access-k25ll\") pod \"monitoring-plugin-85bbbbbf4-72m69\" (UID: \"21d22bca-990d-44c3-967d-1989bf231b5f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.266645 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.266610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:45.344410 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.344286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21d22bca-990d-44c3-967d-1989bf231b5f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-72m69\" (UID: \"21d22bca-990d-44c3-967d-1989bf231b5f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.344410 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.344399 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k25ll\" (UniqueName: \"kubernetes.io/projected/21d22bca-990d-44c3-967d-1989bf231b5f-kube-api-access-k25ll\") pod \"monitoring-plugin-85bbbbbf4-72m69\" (UID: \"21d22bca-990d-44c3-967d-1989bf231b5f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.344701 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:45.344677 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 11:13:45.344859 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:45.344765 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d22bca-990d-44c3-967d-1989bf231b5f-monitoring-plugin-cert podName:21d22bca-990d-44c3-967d-1989bf231b5f nodeName:}" failed. No retries permitted until 2026-09-10 11:13:45.844739155 +0000 UTC m=+64.203795340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/21d22bca-990d-44c3-967d-1989bf231b5f-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-72m69" (UID: "21d22bca-990d-44c3-967d-1989bf231b5f") : secret "monitoring-plugin-cert" not found Sep 10 11:13:45.371753 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.371724 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k25ll\" (UniqueName: \"kubernetes.io/projected/21d22bca-990d-44c3-967d-1989bf231b5f-kube-api-access-k25ll\") pod \"monitoring-plugin-85bbbbbf4-72m69\" (UID: \"21d22bca-990d-44c3-967d-1989bf231b5f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.461227 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.461192 2560 generic.go:356] "Generic (PLEG): container finished" podID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerID="4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5" exitCode=0 Sep 10 11:13:45.461619 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.461269 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5"} Sep 10 11:13:45.464118 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.464093 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" event={"ID":"eb05e7bf-65d9-4b36-a486-34294952a59c","Type":"ContainerStarted","Data":"9422deff2b7969cea54d02498149edecb8536ea5c61ac205451ab08a12683d24"} Sep 10 11:13:45.464228 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.464150 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" event={"ID":"eb05e7bf-65d9-4b36-a486-34294952a59c","Type":"ContainerStarted","Data":"55b9f7c9f96d53c27f3908eb39c93a944c247ccae50cf395339d223f1518d48e"} Sep 10 11:13:45.467555 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.467153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5xtzc" event={"ID":"7d8d509c-426a-4ac0-a3dd-90c2756bff1f","Type":"ContainerStarted","Data":"7ea74b7362b4cf23bab139da5f0ef6fd7c25ee32f7b3cf3584fcd1ab719db439"} Sep 10 11:13:45.467555 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.467198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5xtzc" event={"ID":"7d8d509c-426a-4ac0-a3dd-90c2756bff1f","Type":"ContainerStarted","Data":"51e6517409b6dc1cd7b18ba905121296545c56dae150d56acab54d4815dc880e"} Sep 10 11:13:45.472033 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.472012 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:45.518213 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.518183 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-ff66b8fb-9xtkv"] Sep 10 11:13:45.522895 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.522177 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5xtzc" podStartSLOduration=3.186863633 podStartE2EDuration="5.522154192s" totalImagesPullingTime="2.245612453s" totalInitContainerRuntime="89.678106ms" isStatefulPod=true podCreationTimestamp="2026-09-10 11:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:45.521197011 +0000 UTC m=+63.880253217" watchObservedRunningTime="2026-09-10 11:13:45.522154192 +0000 UTC m=+63.881210394" Sep 10 11:13:45.523167 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:45.523111 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3bc84d1_9c77_4f59_9674_2439981a24af.slice/crio-6ccac0d2d6f46b6eb634e216923d272c9baec96ec784af6584a7be0ecf52c75d WatchSource:0}: Error finding container 6ccac0d2d6f46b6eb634e216923d272c9baec96ec784af6584a7be0ecf52c75d: Status 404 returned error can't find the container with id 6ccac0d2d6f46b6eb634e216923d272c9baec96ec784af6584a7be0ecf52c75d Sep 10 11:13:45.577652 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.577621 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:45.672072 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.672039 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-d45487466-wxmzn"] Sep 10 11:13:45.675659 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.675644 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.678984 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.678965 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 11:13:45.679719 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.679696 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 11:13:45.679835 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.679807 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 11:13:45.683314 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.683277 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 11:13:45.683433 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.683413 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4zckr\"" Sep 10 11:13:45.683693 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.683656 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 11:13:45.689501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.689480 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 11:13:45.696969 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.696946 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d45487466-wxmzn"] Sep 10 11:13:45.738700 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.738673 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:45.749839 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.749818 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-metrics-client-ca\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.749928 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.749851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-federate-client-tls\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.749928 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.749877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.750015 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.749924 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-telemeter-client-tls\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.750015 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.749993 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d764q\" (UniqueName: \"kubernetes.io/projected/a49020fd-3607-4d2c-80ea-7fa2315648af-kube-api-access-d764q\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.750077 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.750044 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-secret-telemeter-client\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.750077 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.750068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.750136 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.750106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-serving-certs-ca-bundle\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.801588 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.801552 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f7759f67-sn6n4"] Sep 10 11:13:45.842853 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.842816 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69c4d4b5f7-4cmrr"] Sep 10 11:13:45.846013 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.845991 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.851068 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851043 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21d22bca-990d-44c3-967d-1989bf231b5f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-72m69\" (UID: \"21d22bca-990d-44c3-967d-1989bf231b5f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.851198 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851087 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-secret-telemeter-client\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.851198 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.851198 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-serving-certs-ca-bundle\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.851375 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-metrics-client-ca\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.851375 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851256 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-federate-client-tls\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.851375 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.851375 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-telemeter-client-tls\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.851557 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.851398 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d764q\" (UniqueName: \"kubernetes.io/projected/a49020fd-3607-4d2c-80ea-7fa2315648af-kube-api-access-d764q\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.852028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.852005 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-metrics-client-ca\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.852582 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.852519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.853025 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.852998 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a49020fd-3607-4d2c-80ea-7fa2315648af-serving-certs-ca-bundle\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.854089 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.854068 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/21d22bca-990d-44c3-967d-1989bf231b5f-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-72m69\" (UID: \"21d22bca-990d-44c3-967d-1989bf231b5f\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:45.854252 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.854226 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-secret-telemeter-client\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.854415 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.854390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.854933 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.854872 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-telemeter-client-tls\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.855122 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.855103 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a49020fd-3607-4d2c-80ea-7fa2315648af-federate-client-tls\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.859790 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.859768 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c4d4b5f7-4cmrr"] Sep 10 11:13:45.861824 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.861799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:45.862442 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.862420 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d764q\" (UniqueName: \"kubernetes.io/projected/a49020fd-3607-4d2c-80ea-7fa2315648af-kube-api-access-d764q\") pod \"telemeter-client-d45487466-wxmzn\" (UID: \"a49020fd-3607-4d2c-80ea-7fa2315648af\") " pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:45.948415 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.948332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:45.952563 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.952542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-config\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.952679 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.952661 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-service-ca\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.952724 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.952694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj8pz\" (UniqueName: \"kubernetes.io/projected/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-kube-api-access-wj8pz\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.952761 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.952744 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-serving-cert\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.952796 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.952759 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-oauth-config\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.952796 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.952781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-oauth-serving-cert\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.952854 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.952807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-trusted-ca-bundle\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:45.987686 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:45.987653 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" Sep 10 11:13:46.054017 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.053981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-service-ca\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054166 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8pz\" (UniqueName: \"kubernetes.io/projected/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-kube-api-access-wj8pz\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054166 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-serving-cert\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054166 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-oauth-config\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054327 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054171 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-oauth-serving-cert\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054327 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-trusted-ca-bundle\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054327 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054238 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-config\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054850 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054813 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-service-ca\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.054993 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.054970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-config\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.055593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.055539 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-oauth-serving-cert\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.056241 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.056195 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-trusted-ca-bundle\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.057803 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.057646 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-oauth-config\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.058191 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.058104 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-serving-cert\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.064986 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.064958 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj8pz\" (UniqueName: \"kubernetes.io/projected/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-kube-api-access-wj8pz\") pod \"console-69c4d4b5f7-4cmrr\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.115776 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.115742 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:46.155761 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.155722 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:46.270983 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.270888 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d45487466-wxmzn"] Sep 10 11:13:46.275444 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:46.275407 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49020fd_3607_4d2c_80ea_7fa2315648af.slice/crio-091d1ea10ecb951faee7e024e06d7ff93518715c6d6c7a436f7adc96d338cea6 WatchSource:0}: Error finding container 091d1ea10ecb951faee7e024e06d7ff93518715c6d6c7a436f7adc96d338cea6: Status 404 returned error can't find the container with id 091d1ea10ecb951faee7e024e06d7ff93518715c6d6c7a436f7adc96d338cea6 Sep 10 11:13:46.398508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.398473 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69"] Sep 10 11:13:46.460069 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.459977 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69c4d4b5f7-4cmrr"] Sep 10 11:13:46.475987 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.475905 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" event={"ID":"a49020fd-3607-4d2c-80ea-7fa2315648af","Type":"ContainerStarted","Data":"091d1ea10ecb951faee7e024e06d7ff93518715c6d6c7a436f7adc96d338cea6"} Sep 10 11:13:46.479484 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.479447 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" event={"ID":"eb05e7bf-65d9-4b36-a486-34294952a59c","Type":"ContainerStarted","Data":"4d3eee76d73440cad90165560b5048ebf3484c66d0ecb84d33cd46fc58ef5c27"} Sep 10 11:13:46.481361 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.481227 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" event={"ID":"b3bc84d1-9c77-4f59-9674-2439981a24af","Type":"ContainerStarted","Data":"6ccac0d2d6f46b6eb634e216923d272c9baec96ec784af6584a7be0ecf52c75d"} Sep 10 11:13:46.549460 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:46.549395 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d22bca_990d_44c3_967d_1989bf231b5f.slice/crio-8b1da7d850861288b8bdb8d85b2e20ced8bbd107d61875330ad0d5af9a012264 WatchSource:0}: Error finding container 8b1da7d850861288b8bdb8d85b2e20ced8bbd107d61875330ad0d5af9a012264: Status 404 returned error can't find the container with id 8b1da7d850861288b8bdb8d85b2e20ced8bbd107d61875330ad0d5af9a012264 Sep 10 11:13:46.550372 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:46.550211 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7df0e61d_a4d5_477d_8235_279c6ddf7dd9.slice/crio-ee992fe3be7ca5254a412cc9949a6b7ba7978f8d2c954e02dd81c119ebd47f72 WatchSource:0}: Error finding container ee992fe3be7ca5254a412cc9949a6b7ba7978f8d2c954e02dd81c119ebd47f72: Status 404 returned error can't find the container with id ee992fe3be7ca5254a412cc9949a6b7ba7978f8d2c954e02dd81c119ebd47f72 Sep 10 11:13:46.655193 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.655162 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:46.751883 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.751799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:46.910316 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.909699 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:13:46.914830 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.914736 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.922548 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.922360 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 11:13:46.922548 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.922387 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 11:13:46.923867 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.923728 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 11:13:46.925423 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.925160 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 11:13:46.925423 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.925209 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 11:13:46.925605 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.925508 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 11:13:46.925988 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.925718 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 11:13:46.925988 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.925788 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 11:13:46.926903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.926441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:46.933076 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.932808 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 11:13:46.933076 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.933086 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 11:13:46.933367 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.933345 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9tr75\"" Sep 10 11:13:46.934426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.934087 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 11:13:46.934426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.934172 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 11:13:46.934574 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.934454 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 11:13:46.934787 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.934646 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-31ju1bg6cghqd\"" Sep 10 11:13:46.955327 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.952772 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:13:46.961863 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.961584 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 11:13:46.963636 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.963535 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.963788 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.963724 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.963876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.963848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.963941 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.963905 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.963941 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.963933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.964039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.964005 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.964718 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.964661 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.964879 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.964858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.965030 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.964971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.965030 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.965007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmcs9\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-kube-api-access-cmcs9\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.965331 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.965285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-web-config\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.965491 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.965464 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.965745 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.965694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.966084 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.966062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.966260 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.966170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config-out\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.966260 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.966233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.966400 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.966281 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.966400 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.966329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.966400 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.966359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:46.966400 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:46.966395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.067903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.067804 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.067903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.067860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config-out\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.068137 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.067920 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.068137 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.067969 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.068137 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.067996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.068137 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.068137 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.068137 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068763 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068787 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068820 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmcs9\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-kube-api-access-cmcs9\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-web-config\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068927 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.068990 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.069666 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.069224 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.070383 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.069806 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.073741 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.070588 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.073741 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.071281 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.073741 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.071774 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.073741 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.072032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.074035 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.073841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.074453 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.074429 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.074648 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.074564 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.076411 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.075067 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.076411 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.075402 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.076411 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.076032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-web-config\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.076411 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.076135 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.076411 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.076355 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config-out\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.076707 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.076468 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.077131 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.076868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.077131 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.076890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.077131 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.077098 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.078958 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.078935 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.084276 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.082359 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmcs9\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-kube-api-access-cmcs9\") pod \"prometheus-k8s-0\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.229831 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.229791 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:47.489763 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.489719 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" event={"ID":"eb05e7bf-65d9-4b36-a486-34294952a59c","Type":"ContainerStarted","Data":"756ec35e8963482c60ec5819d9c84a3350f0ada66d46ce1431f4a38c98d74b1a"} Sep 10 11:13:47.490256 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.489801 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" event={"ID":"eb05e7bf-65d9-4b36-a486-34294952a59c","Type":"ContainerStarted","Data":"a558b92a154bdc3f70a89ec1ee1084a6a25ca318c2c8d7a0c6768dc0ff5394f7"} Sep 10 11:13:47.492208 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.492139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c4d4b5f7-4cmrr" event={"ID":"7df0e61d-a4d5-477d-8235-279c6ddf7dd9","Type":"ContainerStarted","Data":"7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812"} Sep 10 11:13:47.492208 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.492181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c4d4b5f7-4cmrr" event={"ID":"7df0e61d-a4d5-477d-8235-279c6ddf7dd9","Type":"ContainerStarted","Data":"ee992fe3be7ca5254a412cc9949a6b7ba7978f8d2c954e02dd81c119ebd47f72"} Sep 10 11:13:47.493975 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.493914 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" event={"ID":"21d22bca-990d-44c3-967d-1989bf231b5f","Type":"ContainerStarted","Data":"8b1da7d850861288b8bdb8d85b2e20ced8bbd107d61875330ad0d5af9a012264"} Sep 10 11:13:47.518966 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.518892 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69c4d4b5f7-4cmrr" podStartSLOduration=2.518849914 podStartE2EDuration="2.518849914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:47.517195165 +0000 UTC m=+65.876251405" watchObservedRunningTime="2026-09-10 11:13:47.518849914 +0000 UTC m=+65.877906120" Sep 10 11:13:47.522998 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.522968 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:47.611135 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.611049 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:47.977570 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.977535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:47.977762 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.977636 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:47.980862 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.980764 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 11:13:47.980999 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.980781 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 11:13:47.991191 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.991164 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f8581c-3527-40bb-89f6-b51b45a0d4af-metrics-certs\") pod \"network-metrics-daemon-r42wc\" (UID: \"f4f8581c-3527-40bb-89f6-b51b45a0d4af\") " pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:47.991346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:47.991164 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f5e0a42-7fba-460a-b3d2-67a92ca2c94e-original-pull-secret\") pod \"global-pull-secret-syncer-dsmzj\" (UID: \"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e\") " pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:48.078493 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.078400 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:48.081671 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.081638 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 11:13:48.091945 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.091915 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 11:13:48.102537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.102506 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk5qj\" (UniqueName: \"kubernetes.io/projected/6ca7ad53-128f-4ce9-8dc7-ffa5482323c8-kube-api-access-sk5qj\") pod \"network-check-target-9tdqv\" (UID: \"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8\") " pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:48.180720 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.180542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:13:48.182093 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.182063 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qdxwv\"" Sep 10 11:13:48.185397 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.185369 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsmzj" Sep 10 11:13:48.189503 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.189480 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:48.193517 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.193497 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t7cdh\"" Sep 10 11:13:48.201288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.201266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r42wc" Sep 10 11:13:48.240777 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:48.240745 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79aa6da3_d13a_42ad_83e0_eb6e03b5e940.slice/crio-b077e2eeb1709fbc804c5ec3cd62ddcad01f5aeb911da9e5e98a1de0c3e11fa6 WatchSource:0}: Error finding container b077e2eeb1709fbc804c5ec3cd62ddcad01f5aeb911da9e5e98a1de0c3e11fa6: Status 404 returned error can't find the container with id b077e2eeb1709fbc804c5ec3cd62ddcad01f5aeb911da9e5e98a1de0c3e11fa6 Sep 10 11:13:48.502211 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.502168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" event={"ID":"eb05e7bf-65d9-4b36-a486-34294952a59c","Type":"ContainerStarted","Data":"6a6e0290c79aa4a1e6da4883ef16a8ea8785bc31dc5583964e222068d3376a87"} Sep 10 11:13:48.502687 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.502550 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:48.503888 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.503864 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerStarted","Data":"b077e2eeb1709fbc804c5ec3cd62ddcad01f5aeb911da9e5e98a1de0c3e11fa6"} Sep 10 11:13:48.537049 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:48.536985 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" podStartSLOduration=3.479742198 podStartE2EDuration="6.536961634s" totalImagesPullingTime="3.057219436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:48.535220319 +0000 UTC m=+66.894276577" watchObservedRunningTime="2026-09-10 11:13:48.536961634 +0000 UTC m=+66.896017841" Sep 10 11:13:49.029656 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.029556 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9tdqv"] Sep 10 11:13:49.039063 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.036347 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r42wc"] Sep 10 11:13:49.039063 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.036958 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dsmzj"] Sep 10 11:13:49.043807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:49.042107 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f8581c_3527_40bb_89f6_b51b45a0d4af.slice/crio-b68d5c43c9f315d9715f0b46e052ffc0c1a820c8f810c445f590506ff8efbfd1 WatchSource:0}: Error finding container b68d5c43c9f315d9715f0b46e052ffc0c1a820c8f810c445f590506ff8efbfd1: Status 404 returned error can't find the container with id b68d5c43c9f315d9715f0b46e052ffc0c1a820c8f810c445f590506ff8efbfd1 Sep 10 11:13:49.043807 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:13:49.043141 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca7ad53_128f_4ce9_8dc7_ffa5482323c8.slice/crio-c1698a3c3034bb7310b5085ef3a5633044b93d6babecedca679fcaae8563befb WatchSource:0}: Error finding container c1698a3c3034bb7310b5085ef3a5633044b93d6babecedca679fcaae8563befb: Status 404 returned error can't find the container with id c1698a3c3034bb7310b5085ef3a5633044b93d6babecedca679fcaae8563befb Sep 10 11:13:49.512050 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.511958 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerStarted","Data":"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620"} Sep 10 11:13:49.512050 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.512037 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerStarted","Data":"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb"} Sep 10 11:13:49.512564 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.512085 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerStarted","Data":"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d"} Sep 10 11:13:49.512564 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.512127 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerStarted","Data":"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df"} Sep 10 11:13:49.512564 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.512167 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerStarted","Data":"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7"} Sep 10 11:13:49.512564 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.512207 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerStarted","Data":"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1"} Sep 10 11:13:49.514629 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.514598 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" event={"ID":"a49020fd-3607-4d2c-80ea-7fa2315648af","Type":"ContainerStarted","Data":"82523471c97557afc1cf4f3a3d3c85da69a1b49b15ac68d0f26c600c43e44a99"} Sep 10 11:13:49.514756 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.514659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" event={"ID":"a49020fd-3607-4d2c-80ea-7fa2315648af","Type":"ContainerStarted","Data":"9a277676f61259ab617ca29d3fc118ec86233788b52945bf7c6c7e7ab0decbcb"} Sep 10 11:13:49.514756 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.514680 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" event={"ID":"a49020fd-3607-4d2c-80ea-7fa2315648af","Type":"ContainerStarted","Data":"4ab54e0e8e607f9bcb3d80dca68e340df5b078230d9e80195455a36bef25e2c8"} Sep 10 11:13:49.515909 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.515886 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dsmzj" event={"ID":"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e","Type":"ContainerStarted","Data":"19d0d3855d0e6bb14906ac6d6de462f248d1627e33bcb08a3b6b262eccf30367"} Sep 10 11:13:49.517516 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.517489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" event={"ID":"21d22bca-990d-44c3-967d-1989bf231b5f","Type":"ContainerStarted","Data":"221cae54df82b56fcd380d62639491cdf3d3015c53c105922eac5c2f14d84c6d"} Sep 10 11:13:49.517696 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.517654 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:49.522909 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.522888 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" event={"ID":"b3bc84d1-9c77-4f59-9674-2439981a24af","Type":"ContainerStarted","Data":"deb04f9bcf382e8796309ccc920cd95461b3ecc3325fd74959a440c63e772579"} Sep 10 11:13:49.524858 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.524837 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" Sep 10 11:13:49.525320 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.525269 2560 generic.go:356] "Generic (PLEG): container finished" podID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerID="94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117" exitCode=0 Sep 10 11:13:49.525422 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.525343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117"} Sep 10 11:13:49.526642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.526620 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r42wc" event={"ID":"f4f8581c-3527-40bb-89f6-b51b45a0d4af","Type":"ContainerStarted","Data":"b68d5c43c9f315d9715f0b46e052ffc0c1a820c8f810c445f590506ff8efbfd1"} Sep 10 11:13:49.527875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.527854 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9tdqv" event={"ID":"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8","Type":"ContainerStarted","Data":"c1698a3c3034bb7310b5085ef3a5633044b93d6babecedca679fcaae8563befb"} Sep 10 11:13:49.546640 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.546598 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.737424021 podStartE2EDuration="8.546583583s" totalImagesPullingTime="4.793018907s" totalInitContainerRuntime="16.140655ms" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:49.545862924 +0000 UTC m=+67.904919129" watchObservedRunningTime="2026-09-10 11:13:49.546583583 +0000 UTC m=+67.905639788" Sep 10 11:13:49.607337 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.607255 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" podStartSLOduration=2.454735035 podStartE2EDuration="5.6072306s" totalImagesPullingTime="3.152495565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:49.606430047 +0000 UTC m=+67.965486254" watchObservedRunningTime="2026-09-10 11:13:49.6072306 +0000 UTC m=+67.966286806" Sep 10 11:13:49.636996 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.636941 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-d45487466-wxmzn" podStartSLOduration=2.166396184 podStartE2EDuration="4.636923822s" totalImagesPullingTime="2.470527638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:49.636112086 +0000 UTC m=+67.995168327" watchObservedRunningTime="2026-09-10 11:13:49.636923822 +0000 UTC m=+67.995980030" Sep 10 11:13:49.657206 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:49.657148 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-72m69" podStartSLOduration=2.465446262 podStartE2EDuration="4.657127266s" totalImagesPullingTime="2.191681004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:49.654452167 +0000 UTC m=+68.013508375" watchObservedRunningTime="2026-09-10 11:13:49.657127266 +0000 UTC m=+68.016183471" Sep 10 11:13:50.367056 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.367022 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c4d4b5f7-4cmrr"] Sep 10 11:13:50.421311 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.421265 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5449b7878f-lklsl"] Sep 10 11:13:50.424811 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.424781 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.428650 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:13:50.428605 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-5449b7878f-lklsl\" is forbidden: User \"system:node:ip-10-0-131-79.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-131-79.ec2.internal' and this object" podUID="1d5972ea-3aef-48a5-9b86-4ecf50839f7a" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.439950 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.439924 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5449b7878f-lklsl"] Sep 10 11:13:50.505936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.505539 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-service-ca\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.505936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.505603 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-config\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.505936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.505630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjxbz\" (UniqueName: \"kubernetes.io/projected/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-kube-api-access-gjxbz\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.505936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.505695 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-trusted-ca-bundle\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.505936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.505769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-oauth-serving-cert\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.505936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.505802 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-serving-cert\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.505936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.505828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-oauth-config\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.609282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.607110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-trusted-ca-bundle\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.609282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.607376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-oauth-serving-cert\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.609282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.607450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-serving-cert\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.609282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.607508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-oauth-config\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.609282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.607594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-service-ca\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.609282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.607697 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-config\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.609282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.607725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjxbz\" (UniqueName: \"kubernetes.io/projected/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-kube-api-access-gjxbz\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.611755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.609837 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-trusted-ca-bundle\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.611755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.610333 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-oauth-serving-cert\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.611755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.610760 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-config\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.611755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.611710 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-service-ca\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.615810 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.615788 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-oauth-config\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.616415 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.616371 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-serving-cert\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.618141 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.618059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjxbz\" (UniqueName: \"kubernetes.io/projected/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-kube-api-access-gjxbz\") pod \"console-5449b7878f-lklsl\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.738897 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.738680 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:13:50.830432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.830110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:50.920220 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:50.920184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:51.045263 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.044807 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5449b7878f-lklsl"] Sep 10 11:13:51.545361 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.544631 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5449b7878f-lklsl" event={"ID":"1d5972ea-3aef-48a5-9b86-4ecf50839f7a","Type":"ContainerStarted","Data":"0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474"} Sep 10 11:13:51.545361 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.544695 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5449b7878f-lklsl" event={"ID":"1d5972ea-3aef-48a5-9b86-4ecf50839f7a","Type":"ContainerStarted","Data":"52191fe5a4a6ff2a933208b895e36ecb43916deedbbf403dc8ba6a468a0cb995"} Sep 10 11:13:51.550536 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.549947 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r42wc" event={"ID":"f4f8581c-3527-40bb-89f6-b51b45a0d4af","Type":"ContainerStarted","Data":"178d77b58ade804bb5fb95057511d405804240c7973545994bcf43e09fb8e920"} Sep 10 11:13:51.550536 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.549998 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r42wc" event={"ID":"f4f8581c-3527-40bb-89f6-b51b45a0d4af","Type":"ContainerStarted","Data":"020c2dc28b3f8ed0479d15a57ac9429d77ae90766d8e1e07a064c16c3eac78c8"} Sep 10 11:13:51.573868 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.573804 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5449b7878f-lklsl" podStartSLOduration=1.573783076 podStartE2EDuration="1.573783076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:51.573370834 +0000 UTC m=+69.932427041" watchObservedRunningTime="2026-09-10 11:13:51.573783076 +0000 UTC m=+69.932839284" Sep 10 11:13:51.599096 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.598593 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r42wc" podStartSLOduration=68.180728124 podStartE2EDuration="1m9.598567412s" totalImagesPullingTime="1.417839288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:51.597851661 +0000 UTC m=+69.956907887" watchObservedRunningTime="2026-09-10 11:13:51.598567412 +0000 UTC m=+69.957623622" Sep 10 11:13:51.668961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.668829 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:13:51.721830 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.721743 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:51.818310 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:51.818218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:53.181772 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:53.181744 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:53.257502 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:53.257433 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:54.535214 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:54.535178 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-74f774c495-5crkk" Sep 10 11:13:56.156944 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.156743 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:13:56.578000 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.577895 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dsmzj" event={"ID":"7f5e0a42-7fba-460a-b3d2-67a92ca2c94e","Type":"ContainerStarted","Data":"7a130164b7206f411ee2c8c7be20ed26dd36171e1e3669bce8d784391489b30a"} Sep 10 11:13:56.581427 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.581389 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerStarted","Data":"eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f"} Sep 10 11:13:56.581572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.581460 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerStarted","Data":"9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78"} Sep 10 11:13:56.581572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.581489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerStarted","Data":"276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70"} Sep 10 11:13:56.581572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.581528 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerStarted","Data":"66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74"} Sep 10 11:13:56.581572 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.581556 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerStarted","Data":"ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c"} Sep 10 11:13:56.581749 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.581579 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerStarted","Data":"898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530"} Sep 10 11:13:56.582828 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.582810 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9tdqv" event={"ID":"6ca7ad53-128f-4ce9-8dc7-ffa5482323c8","Type":"ContainerStarted","Data":"54dd60490cf9e006ce813992a0a02c2011d42871e4ef238c6dc7f6353bdbaa50"} Sep 10 11:13:56.582944 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.582932 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:13:56.612290 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.612241 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dsmzj" podStartSLOduration=67.875681233 podStartE2EDuration="1m14.612224542s" totalImagesPullingTime="6.736543309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:56.609659321 +0000 UTC m=+74.968715527" watchObservedRunningTime="2026-09-10 11:13:56.612224542 +0000 UTC m=+74.971280747" Sep 10 11:13:56.718107 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.718051 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.427265105 podStartE2EDuration="10.718032912s" totalImagesPullingTime="6.253243952s" totalInitContainerRuntime="37.523855ms" isStatefulPod=false podCreationTimestamp="2026-09-10 11:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:56.715704197 +0000 UTC m=+75.074760403" watchObservedRunningTime="2026-09-10 11:13:56.718032912 +0000 UTC m=+75.077089123" Sep 10 11:13:56.741870 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.741832 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:56.813743 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.813692 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9tdqv" podStartSLOduration=68.087995148 podStartE2EDuration="1m14.813673236s" totalImagesPullingTime="6.725678088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:13:56.813179432 +0000 UTC m=+75.172235638" watchObservedRunningTime="2026-09-10 11:13:56.813673236 +0000 UTC m=+75.172729470" Sep 10 11:13:56.903950 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:56.903851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:57.230788 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:57.230751 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:13:57.863729 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:57.863694 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:13:57.979382 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:13:57.979332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:00.095352 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:00.095317 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:00.193190 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:00.193145 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:00.739801 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:00.739761 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:14:00.739969 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:00.739832 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:14:00.744712 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:00.744688 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:14:01.610391 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:01.610362 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:14:01.670438 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:01.670402 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c45779ddc-qtqkm"] Sep 10 11:14:01.825817 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:01.825780 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:01.944962 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:01.944925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:02.026499 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:02.026469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:05.039665 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:05.039519 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:05.140110 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:05.140074 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:05.164175 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:05.164141 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:14:05.164344 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:05.164228 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:14:05.214617 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:05.214576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:10.821240 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:10.821167 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65f7759f67-sn6n4" podUID="628440b1-3553-4e6a-a46c-25482c0bac94" containerName="console" containerID="cri-o://0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be" gracePeriod=15 Sep 10 11:14:11.064474 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.064443 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:14:11.211110 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211074 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-console-config\") pod \"628440b1-3553-4e6a-a46c-25482c0bac94\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " Sep 10 11:14:11.211264 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211124 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-serving-cert\") pod \"628440b1-3553-4e6a-a46c-25482c0bac94\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " Sep 10 11:14:11.211264 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211157 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-trusted-ca-bundle\") pod \"628440b1-3553-4e6a-a46c-25482c0bac94\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " Sep 10 11:14:11.211369 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211352 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-oauth-config\") pod \"628440b1-3553-4e6a-a46c-25482c0bac94\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " Sep 10 11:14:11.211416 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211396 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-oauth-serving-cert\") pod \"628440b1-3553-4e6a-a46c-25482c0bac94\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " Sep 10 11:14:11.211452 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211429 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-console-config" (OuterVolumeSpecName: "console-config") pod "628440b1-3553-4e6a-a46c-25482c0bac94" (UID: "628440b1-3553-4e6a-a46c-25482c0bac94"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:11.211502 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211477 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-service-ca\") pod \"628440b1-3553-4e6a-a46c-25482c0bac94\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " Sep 10 11:14:11.211554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211533 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzbjp\" (UniqueName: \"kubernetes.io/projected/628440b1-3553-4e6a-a46c-25482c0bac94-kube-api-access-xzbjp\") pod \"628440b1-3553-4e6a-a46c-25482c0bac94\" (UID: \"628440b1-3553-4e6a-a46c-25482c0bac94\") " Sep 10 11:14:11.211611 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211561 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "628440b1-3553-4e6a-a46c-25482c0bac94" (UID: "628440b1-3553-4e6a-a46c-25482c0bac94"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:11.211829 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211777 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-service-ca" (OuterVolumeSpecName: "service-ca") pod "628440b1-3553-4e6a-a46c-25482c0bac94" (UID: "628440b1-3553-4e6a-a46c-25482c0bac94"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:11.211829 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211780 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "628440b1-3553-4e6a-a46c-25482c0bac94" (UID: "628440b1-3553-4e6a-a46c-25482c0bac94"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:11.211957 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211833 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-console-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:11.211957 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.211850 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-trusted-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:11.213791 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.213766 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628440b1-3553-4e6a-a46c-25482c0bac94-kube-api-access-xzbjp" (OuterVolumeSpecName: "kube-api-access-xzbjp") pod "628440b1-3553-4e6a-a46c-25482c0bac94" (UID: "628440b1-3553-4e6a-a46c-25482c0bac94"). InnerVolumeSpecName "kube-api-access-xzbjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:14:11.213900 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.213872 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "628440b1-3553-4e6a-a46c-25482c0bac94" (UID: "628440b1-3553-4e6a-a46c-25482c0bac94"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:14:11.213950 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.213931 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "628440b1-3553-4e6a-a46c-25482c0bac94" (UID: "628440b1-3553-4e6a-a46c-25482c0bac94"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:14:11.312593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.312530 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-service-ca\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:11.312593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.312582 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzbjp\" (UniqueName: \"kubernetes.io/projected/628440b1-3553-4e6a-a46c-25482c0bac94-kube-api-access-xzbjp\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:11.312593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.312596 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:11.312593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.312609 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/628440b1-3553-4e6a-a46c-25482c0bac94-console-oauth-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:11.312910 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.312623 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/628440b1-3553-4e6a-a46c-25482c0bac94-oauth-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:11.649248 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.649170 2560 generic.go:356] "Generic (PLEG): container finished" podID="628440b1-3553-4e6a-a46c-25482c0bac94" containerID="0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be" exitCode=0 Sep 10 11:14:11.649248 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.649214 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7759f67-sn6n4" Sep 10 11:14:11.649509 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.649257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7759f67-sn6n4" event={"ID":"628440b1-3553-4e6a-a46c-25482c0bac94","Type":"ContainerDied","Data":"0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be"} Sep 10 11:14:11.649509 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.649337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7759f67-sn6n4" event={"ID":"628440b1-3553-4e6a-a46c-25482c0bac94","Type":"ContainerDied","Data":"268e517565287b910d8c6a2239a53f052efd66fe03c517624cfc8f1bfb5372a7"} Sep 10 11:14:11.649509 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.649378 2560 scope.go:117] "RemoveContainer" containerID="0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be" Sep 10 11:14:11.658468 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.658450 2560 scope.go:117] "RemoveContainer" containerID="0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be" Sep 10 11:14:11.658749 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:14:11.658731 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be\": container with ID starting with 0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be not found: ID does not exist" containerID="0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be" Sep 10 11:14:11.658815 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.658762 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be"} err="failed to get container status \"0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be\": rpc error: code = NotFound desc = could not find container \"0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be\": container with ID starting with 0869330d08feb5e593fb5b5cd0cb9751f35bc9c26855a657b79a665ff908f4be not found: ID does not exist" Sep 10 11:14:11.673409 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.673386 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f7759f67-sn6n4"] Sep 10 11:14:11.679642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:11.679620 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65f7759f67-sn6n4"] Sep 10 11:14:12.172701 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:12.172666 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628440b1-3553-4e6a-a46c-25482c0bac94" path="/var/lib/kubelet/pods/628440b1-3553-4e6a-a46c-25482c0bac94/volumes" Sep 10 11:14:12.855266 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:12.855232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:12.945394 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:12.945365 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:15.398425 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.398386 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69c4d4b5f7-4cmrr" podUID="7df0e61d-a4d5-477d-8235-279c6ddf7dd9" containerName="console" containerID="cri-o://7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812" gracePeriod=15 Sep 10 11:14:15.642519 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.642495 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:14:15.646511 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646491 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-oauth-serving-cert\") pod \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " Sep 10 11:14:15.646579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646534 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-service-ca\") pod \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " Sep 10 11:14:15.646579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646558 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-trusted-ca-bundle\") pod \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " Sep 10 11:14:15.646663 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646612 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-config\") pod \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " Sep 10 11:14:15.646663 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646638 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj8pz\" (UniqueName: \"kubernetes.io/projected/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-kube-api-access-wj8pz\") pod \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " Sep 10 11:14:15.646760 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646666 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-oauth-config\") pod \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " Sep 10 11:14:15.646760 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646711 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-serving-cert\") pod \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\" (UID: \"7df0e61d-a4d5-477d-8235-279c6ddf7dd9\") " Sep 10 11:14:15.646907 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646881 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7df0e61d-a4d5-477d-8235-279c6ddf7dd9" (UID: "7df0e61d-a4d5-477d-8235-279c6ddf7dd9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:15.646976 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646948 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7df0e61d-a4d5-477d-8235-279c6ddf7dd9" (UID: "7df0e61d-a4d5-477d-8235-279c6ddf7dd9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:15.647038 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.646957 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-config" (OuterVolumeSpecName: "console-config") pod "7df0e61d-a4d5-477d-8235-279c6ddf7dd9" (UID: "7df0e61d-a4d5-477d-8235-279c6ddf7dd9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:15.647090 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.647048 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-oauth-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:15.647090 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.647066 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-trusted-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:15.647317 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.647280 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-service-ca" (OuterVolumeSpecName: "service-ca") pod "7df0e61d-a4d5-477d-8235-279c6ddf7dd9" (UID: "7df0e61d-a4d5-477d-8235-279c6ddf7dd9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:15.648965 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.648908 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-kube-api-access-wj8pz" (OuterVolumeSpecName: "kube-api-access-wj8pz") pod "7df0e61d-a4d5-477d-8235-279c6ddf7dd9" (UID: "7df0e61d-a4d5-477d-8235-279c6ddf7dd9"). InnerVolumeSpecName "kube-api-access-wj8pz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:14:15.648965 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.648918 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7df0e61d-a4d5-477d-8235-279c6ddf7dd9" (UID: "7df0e61d-a4d5-477d-8235-279c6ddf7dd9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:14:15.649117 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.648997 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7df0e61d-a4d5-477d-8235-279c6ddf7dd9" (UID: "7df0e61d-a4d5-477d-8235-279c6ddf7dd9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:14:15.669588 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.669556 2560 generic.go:356] "Generic (PLEG): container finished" podID="7df0e61d-a4d5-477d-8235-279c6ddf7dd9" containerID="7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812" exitCode=0 Sep 10 11:14:15.669722 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.669608 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69c4d4b5f7-4cmrr" Sep 10 11:14:15.669722 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.669647 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c4d4b5f7-4cmrr" event={"ID":"7df0e61d-a4d5-477d-8235-279c6ddf7dd9","Type":"ContainerDied","Data":"7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812"} Sep 10 11:14:15.669722 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.669717 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69c4d4b5f7-4cmrr" event={"ID":"7df0e61d-a4d5-477d-8235-279c6ddf7dd9","Type":"ContainerDied","Data":"ee992fe3be7ca5254a412cc9949a6b7ba7978f8d2c954e02dd81c119ebd47f72"} Sep 10 11:14:15.669881 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.669746 2560 scope.go:117] "RemoveContainer" containerID="7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812" Sep 10 11:14:15.681988 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.681956 2560 scope.go:117] "RemoveContainer" containerID="7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812" Sep 10 11:14:15.682446 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:14:15.682424 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812\": container with ID starting with 7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812 not found: ID does not exist" containerID="7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812" Sep 10 11:14:15.682516 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.682453 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812"} err="failed to get container status \"7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812\": rpc error: code = NotFound desc = could not find container \"7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812\": container with ID starting with 7034ca9af2578bf77805b00dda01f4bc6f93646a4999dffc24e61878470cb812 not found: ID does not exist" Sep 10 11:14:15.695370 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.695344 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69c4d4b5f7-4cmrr"] Sep 10 11:14:15.699340 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.699322 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69c4d4b5f7-4cmrr"] Sep 10 11:14:15.748100 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.748062 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:15.748100 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.748094 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-service-ca\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:15.748100 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.748109 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:15.748362 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.748120 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj8pz\" (UniqueName: \"kubernetes.io/projected/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-kube-api-access-wj8pz\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:15.748362 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:15.748132 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7df0e61d-a4d5-477d-8235-279c6ddf7dd9-console-oauth-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:16.171143 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:16.171101 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7df0e61d-a4d5-477d-8235-279c6ddf7dd9" path="/var/lib/kubelet/pods/7df0e61d-a4d5-477d-8235-279c6ddf7dd9/volumes" Sep 10 11:14:25.169652 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:25.169619 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:14:25.173766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:25.173742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-ff66b8fb-9xtkv" Sep 10 11:14:26.690497 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:26.690432 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c45779ddc-qtqkm" podUID="a4dda299-2891-4b1e-911b-97551f4e0cd4" containerName="console" containerID="cri-o://88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1" gracePeriod=15 Sep 10 11:14:26.980729 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:26.980694 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:14:27.054596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.054557 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-oauth-serving-cert\") pod \"a4dda299-2891-4b1e-911b-97551f4e0cd4\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " Sep 10 11:14:27.054832 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.054620 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-config\") pod \"a4dda299-2891-4b1e-911b-97551f4e0cd4\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " Sep 10 11:14:27.054832 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.054663 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-serving-cert\") pod \"a4dda299-2891-4b1e-911b-97551f4e0cd4\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " Sep 10 11:14:27.054832 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.054697 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kc7w\" (UniqueName: \"kubernetes.io/projected/a4dda299-2891-4b1e-911b-97551f4e0cd4-kube-api-access-4kc7w\") pod \"a4dda299-2891-4b1e-911b-97551f4e0cd4\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " Sep 10 11:14:27.054832 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.054724 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-oauth-config\") pod \"a4dda299-2891-4b1e-911b-97551f4e0cd4\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " Sep 10 11:14:27.054832 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.054759 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-service-ca\") pod \"a4dda299-2891-4b1e-911b-97551f4e0cd4\" (UID: \"a4dda299-2891-4b1e-911b-97551f4e0cd4\") " Sep 10 11:14:27.055147 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.055030 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-config" (OuterVolumeSpecName: "console-config") pod "a4dda299-2891-4b1e-911b-97551f4e0cd4" (UID: "a4dda299-2891-4b1e-911b-97551f4e0cd4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:27.055147 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.055109 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a4dda299-2891-4b1e-911b-97551f4e0cd4" (UID: "a4dda299-2891-4b1e-911b-97551f4e0cd4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:27.055214 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.055191 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-service-ca" (OuterVolumeSpecName: "service-ca") pod "a4dda299-2891-4b1e-911b-97551f4e0cd4" (UID: "a4dda299-2891-4b1e-911b-97551f4e0cd4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:14:27.057020 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.056987 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a4dda299-2891-4b1e-911b-97551f4e0cd4" (UID: "a4dda299-2891-4b1e-911b-97551f4e0cd4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:14:27.057114 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.057038 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4dda299-2891-4b1e-911b-97551f4e0cd4-kube-api-access-4kc7w" (OuterVolumeSpecName: "kube-api-access-4kc7w") pod "a4dda299-2891-4b1e-911b-97551f4e0cd4" (UID: "a4dda299-2891-4b1e-911b-97551f4e0cd4"). InnerVolumeSpecName "kube-api-access-4kc7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:14:27.057114 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.057052 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a4dda299-2891-4b1e-911b-97551f4e0cd4" (UID: "a4dda299-2891-4b1e-911b-97551f4e0cd4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:14:27.155530 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.155489 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:27.155530 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.155522 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:27.155530 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.155533 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kc7w\" (UniqueName: \"kubernetes.io/projected/a4dda299-2891-4b1e-911b-97551f4e0cd4-kube-api-access-4kc7w\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:27.155530 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.155543 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4dda299-2891-4b1e-911b-97551f4e0cd4-console-oauth-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:27.155786 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.155552 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-service-ca\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:27.155786 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.155560 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4dda299-2891-4b1e-911b-97551f4e0cd4-oauth-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:14:27.590974 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.590946 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9tdqv" Sep 10 11:14:27.727644 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.727605 2560 generic.go:356] "Generic (PLEG): container finished" podID="a4dda299-2891-4b1e-911b-97551f4e0cd4" containerID="88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1" exitCode=0 Sep 10 11:14:27.728043 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.727653 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c45779ddc-qtqkm" Sep 10 11:14:27.728043 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.727675 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c45779ddc-qtqkm" event={"ID":"a4dda299-2891-4b1e-911b-97551f4e0cd4","Type":"ContainerDied","Data":"88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1"} Sep 10 11:14:27.728043 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.727726 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c45779ddc-qtqkm" event={"ID":"a4dda299-2891-4b1e-911b-97551f4e0cd4","Type":"ContainerDied","Data":"fa49abe9f0b2129a8c56653b3aea1a7cb08c56be9c4f730b349b2d01bf019935"} Sep 10 11:14:27.728043 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.727753 2560 scope.go:117] "RemoveContainer" containerID="88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1" Sep 10 11:14:27.736472 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.736451 2560 scope.go:117] "RemoveContainer" containerID="88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1" Sep 10 11:14:27.736771 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:14:27.736755 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1\": container with ID starting with 88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1 not found: ID does not exist" containerID="88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1" Sep 10 11:14:27.736818 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.736779 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1"} err="failed to get container status \"88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1\": rpc error: code = NotFound desc = could not find container \"88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1\": container with ID starting with 88205e3866636dff88b00d21353c91c79b2fcd25f720026558c69656b8dba5a1 not found: ID does not exist" Sep 10 11:14:27.751055 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.751016 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c45779ddc-qtqkm"] Sep 10 11:14:27.758578 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:27.758546 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c45779ddc-qtqkm"] Sep 10 11:14:28.171498 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:28.171463 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4dda299-2891-4b1e-911b-97551f4e0cd4" path="/var/lib/kubelet/pods/a4dda299-2891-4b1e-911b-97551f4e0cd4/volumes" Sep 10 11:14:47.230552 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:47.230520 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:14:47.251195 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:47.251165 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:14:47.832764 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:47.832738 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:14:56.685826 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:56.685788 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:14:56.762126 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:14:56.762095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:00.997408 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:00.997365 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:15:00.997864 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:00.997750 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="alertmanager" containerID="cri-o://3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1" gracePeriod=120 Sep 10 11:15:00.997914 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:00.997847 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-metric" containerID="cri-o://1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb" gracePeriod=120 Sep 10 11:15:00.997914 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:00.997902 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy" containerID="cri-o://9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d" gracePeriod=120 Sep 10 11:15:00.997991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:00.997931 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="config-reloader" containerID="cri-o://caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7" gracePeriod=120 Sep 10 11:15:00.997991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:00.997863 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-web" containerID="cri-o://e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df" gracePeriod=120 Sep 10 11:15:00.998080 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:00.998045 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="prom-label-proxy" containerID="cri-o://4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620" gracePeriod=120 Sep 10 11:15:01.874414 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874382 2560 generic.go:356] "Generic (PLEG): container finished" podID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerID="4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620" exitCode=0 Sep 10 11:15:01.874414 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874408 2560 generic.go:356] "Generic (PLEG): container finished" podID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerID="9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d" exitCode=0 Sep 10 11:15:01.874414 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874414 2560 generic.go:356] "Generic (PLEG): container finished" podID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerID="caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7" exitCode=0 Sep 10 11:15:01.874414 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874420 2560 generic.go:356] "Generic (PLEG): container finished" podID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerID="3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1" exitCode=0 Sep 10 11:15:01.874691 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874462 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620"} Sep 10 11:15:01.874691 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874530 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d"} Sep 10 11:15:01.874691 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7"} Sep 10 11:15:01.874691 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:01.874581 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1"} Sep 10 11:15:02.347231 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.347207 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.364884 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.364852 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-trusted-ca-bundle\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365044 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.364891 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-cluster-tls-config\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365044 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.364943 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-out\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365044 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.364976 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365044 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-volume\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365044 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365041 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-main-db\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365406 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365337 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 11:15:02.365406 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365347 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365406 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365351 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:02.365406 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365392 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-metrics-client-ca\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365617 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365422 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365617 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365473 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdqdk\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-kube-api-access-jdqdk\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365617 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365508 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-web-config\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365617 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365548 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-tls-assets\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365617 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365597 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-web\") pod \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\" (UID: \"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934\") " Sep 10 11:15:02.365912 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365893 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-main-db\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.365976 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.365918 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.367931 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.367897 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:02.369098 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.369070 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:02.369576 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.369518 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-volume" (OuterVolumeSpecName: "config-volume") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:02.369776 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.369752 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-kube-api-access-jdqdk" (OuterVolumeSpecName: "kube-api-access-jdqdk") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "kube-api-access-jdqdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:15:02.369915 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.369887 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:02.370046 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.370015 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:02.370208 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.370188 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:15:02.370369 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.370353 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:02.370534 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.370510 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-out" (OuterVolumeSpecName: "config-out") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 11:15:02.373518 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.373480 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:02.382109 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.382076 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-web-config" (OuterVolumeSpecName: "web-config") pod "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" (UID: "ccf8c940-fc81-4ccc-a4e0-a29d8dac0934"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466492 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-main-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466523 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-metrics-client-ca\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466534 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466544 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdqdk\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-kube-api-access-jdqdk\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466554 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-web-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466563 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-tls-assets\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466571 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466580 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-cluster-tls-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466588 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-out\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.466587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466598 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.467182 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.466606 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934-config-volume\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:02.882327 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.882223 2560 generic.go:356] "Generic (PLEG): container finished" podID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerID="1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb" exitCode=0 Sep 10 11:15:02.882327 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.882249 2560 generic.go:356] "Generic (PLEG): container finished" podID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerID="e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df" exitCode=0 Sep 10 11:15:02.882327 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.882289 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.882535 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.882321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb"} Sep 10 11:15:02.882535 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.882394 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df"} Sep 10 11:15:02.882535 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.882426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccf8c940-fc81-4ccc-a4e0-a29d8dac0934","Type":"ContainerDied","Data":"21c603977e05ec0b964fcd0dc4f7e75c07f0d387643fe6a5687743d860f3bea2"} Sep 10 11:15:02.882535 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.882462 2560 scope.go:117] "RemoveContainer" containerID="4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620" Sep 10 11:15:02.890417 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.890395 2560 scope.go:117] "RemoveContainer" containerID="1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb" Sep 10 11:15:02.897446 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.897425 2560 scope.go:117] "RemoveContainer" containerID="9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d" Sep 10 11:15:02.904259 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.904243 2560 scope.go:117] "RemoveContainer" containerID="e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df" Sep 10 11:15:02.909980 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.909957 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:15:02.915234 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.915200 2560 scope.go:117] "RemoveContainer" containerID="caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7" Sep 10 11:15:02.917028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.917009 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:15:02.923011 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.922993 2560 scope.go:117] "RemoveContainer" containerID="3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1" Sep 10 11:15:02.930455 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.930435 2560 scope.go:117] "RemoveContainer" containerID="4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5" Sep 10 11:15:02.937664 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.937646 2560 scope.go:117] "RemoveContainer" containerID="4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620" Sep 10 11:15:02.937960 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:02.937935 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620\": container with ID starting with 4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620 not found: ID does not exist" containerID="4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620" Sep 10 11:15:02.938086 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.937965 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620"} err="failed to get container status \"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620\": rpc error: code = NotFound desc = could not find container \"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620\": container with ID starting with 4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620 not found: ID does not exist" Sep 10 11:15:02.938086 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.937986 2560 scope.go:117] "RemoveContainer" containerID="1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb" Sep 10 11:15:02.938279 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:02.938263 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb\": container with ID starting with 1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb not found: ID does not exist" containerID="1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb" Sep 10 11:15:02.938332 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.938285 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb"} err="failed to get container status \"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb\": rpc error: code = NotFound desc = could not find container \"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb\": container with ID starting with 1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb not found: ID does not exist" Sep 10 11:15:02.938332 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.938327 2560 scope.go:117] "RemoveContainer" containerID="9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d" Sep 10 11:15:02.938577 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:02.938559 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d\": container with ID starting with 9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d not found: ID does not exist" containerID="9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d" Sep 10 11:15:02.938633 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.938579 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d"} err="failed to get container status \"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d\": rpc error: code = NotFound desc = could not find container \"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d\": container with ID starting with 9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d not found: ID does not exist" Sep 10 11:15:02.938633 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.938593 2560 scope.go:117] "RemoveContainer" containerID="e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df" Sep 10 11:15:02.938828 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:02.938813 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df\": container with ID starting with e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df not found: ID does not exist" containerID="e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df" Sep 10 11:15:02.938867 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.938832 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df"} err="failed to get container status \"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df\": rpc error: code = NotFound desc = could not find container \"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df\": container with ID starting with e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df not found: ID does not exist" Sep 10 11:15:02.938867 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.938845 2560 scope.go:117] "RemoveContainer" containerID="caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7" Sep 10 11:15:02.939070 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:02.939052 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7\": container with ID starting with caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7 not found: ID does not exist" containerID="caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7" Sep 10 11:15:02.939112 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939072 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7"} err="failed to get container status \"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7\": rpc error: code = NotFound desc = could not find container \"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7\": container with ID starting with caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7 not found: ID does not exist" Sep 10 11:15:02.939112 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939085 2560 scope.go:117] "RemoveContainer" containerID="3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1" Sep 10 11:15:02.939291 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:02.939276 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1\": container with ID starting with 3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1 not found: ID does not exist" containerID="3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1" Sep 10 11:15:02.939351 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939294 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1"} err="failed to get container status \"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1\": rpc error: code = NotFound desc = could not find container \"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1\": container with ID starting with 3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1 not found: ID does not exist" Sep 10 11:15:02.939351 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939333 2560 scope.go:117] "RemoveContainer" containerID="4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5" Sep 10 11:15:02.939546 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:02.939530 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5\": container with ID starting with 4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5 not found: ID does not exist" containerID="4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5" Sep 10 11:15:02.939591 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939548 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5"} err="failed to get container status \"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5\": rpc error: code = NotFound desc = could not find container \"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5\": container with ID starting with 4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5 not found: ID does not exist" Sep 10 11:15:02.939591 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939559 2560 scope.go:117] "RemoveContainer" containerID="4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620" Sep 10 11:15:02.939768 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939753 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620"} err="failed to get container status \"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620\": rpc error: code = NotFound desc = could not find container \"4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620\": container with ID starting with 4f6e9ee42b9badd4a752908c4b093b3977b163dd9fcdf33b1bf19ac63c99d620 not found: ID does not exist" Sep 10 11:15:02.939814 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939768 2560 scope.go:117] "RemoveContainer" containerID="1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb" Sep 10 11:15:02.939967 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939951 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb"} err="failed to get container status \"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb\": rpc error: code = NotFound desc = could not find container \"1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb\": container with ID starting with 1e2dab842dd58d382456a1fe2191c271a29740d81b34b52bd028980892f592fb not found: ID does not exist" Sep 10 11:15:02.940007 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.939967 2560 scope.go:117] "RemoveContainer" containerID="9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d" Sep 10 11:15:02.940173 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940154 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d"} err="failed to get container status \"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d\": rpc error: code = NotFound desc = could not find container \"9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d\": container with ID starting with 9458c58e01ea205d0672e4838bd600b0d13ee4589e407065d44e7b995c99647d not found: ID does not exist" Sep 10 11:15:02.940173 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940172 2560 scope.go:117] "RemoveContainer" containerID="e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df" Sep 10 11:15:02.940435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940418 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df"} err="failed to get container status \"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df\": rpc error: code = NotFound desc = could not find container \"e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df\": container with ID starting with e52a972729dd546acfd6f75c0cd3f2e952c764e0052c610ba04e92e2767fc2df not found: ID does not exist" Sep 10 11:15:02.940435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940435 2560 scope.go:117] "RemoveContainer" containerID="caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7" Sep 10 11:15:02.940645 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940628 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7"} err="failed to get container status \"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7\": rpc error: code = NotFound desc = could not find container \"caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7\": container with ID starting with caffc0d56725b130e5aad1bc3b499bd842dfbea5bd6296faecf003939b5acdc7 not found: ID does not exist" Sep 10 11:15:02.940645 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940645 2560 scope.go:117] "RemoveContainer" containerID="3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1" Sep 10 11:15:02.940860 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940845 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1"} err="failed to get container status \"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1\": rpc error: code = NotFound desc = could not find container \"3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1\": container with ID starting with 3a9b4e2f3fc8e90791760f77357446b08159097456baf6b433e3d6e32b711cd1 not found: ID does not exist" Sep 10 11:15:02.940860 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.940859 2560 scope.go:117] "RemoveContainer" containerID="4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5" Sep 10 11:15:02.941087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.941034 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5"} err="failed to get container status \"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5\": rpc error: code = NotFound desc = could not find container \"4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5\": container with ID starting with 4ed8145cbc05f9ec0c43b6535c0e9056e7677c02272f04667282327bb3ca04a5 not found: ID does not exist" Sep 10 11:15:02.954265 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954228 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:15:02.954592 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954575 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="prom-label-proxy" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954597 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="prom-label-proxy" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954609 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="init-config-reloader" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954615 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="init-config-reloader" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954622 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7df0e61d-a4d5-477d-8235-279c6ddf7dd9" containerName="console" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954628 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7df0e61d-a4d5-477d-8235-279c6ddf7dd9" containerName="console" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954635 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-metric" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954640 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-metric" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954650 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="alertmanager" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954657 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="alertmanager" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954669 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4dda299-2891-4b1e-911b-97551f4e0cd4" containerName="console" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954675 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4dda299-2891-4b1e-911b-97551f4e0cd4" containerName="console" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954680 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="config-reloader" Sep 10 11:15:02.954681 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954685 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="config-reloader" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954693 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628440b1-3553-4e6a-a46c-25482c0bac94" containerName="console" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954698 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="628440b1-3553-4e6a-a46c-25482c0bac94" containerName="console" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954705 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-web" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954711 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-web" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954718 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954724 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954777 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954788 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-metric" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954794 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7df0e61d-a4d5-477d-8235-279c6ddf7dd9" containerName="console" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954799 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="prom-label-proxy" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954804 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="628440b1-3553-4e6a-a46c-25482c0bac94" containerName="console" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954809 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4dda299-2891-4b1e-911b-97551f4e0cd4" containerName="console" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954816 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="alertmanager" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954826 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="config-reloader" Sep 10 11:15:02.955087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.954833 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" containerName="kube-rbac-proxy-web" Sep 10 11:15:02.960937 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.960906 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.965083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965056 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 11:15:02.965217 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965092 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 11:15:02.965217 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965125 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5zss5\"" Sep 10 11:15:02.965217 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965135 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 11:15:02.965217 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965150 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 11:15:02.965456 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965441 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 11:15:02.965521 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965443 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 11:15:02.965521 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965443 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 11:15:02.965702 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.965688 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 11:15:02.971140 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971248 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971150 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8c02dba-388a-47c1-ae01-38bd1da194fd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971248 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971248 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8c02dba-388a-47c1-ae01-38bd1da194fd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971276 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8c02dba-388a-47c1-ae01-38bd1da194fd-config-out\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971377 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8c02dba-388a-47c1-ae01-38bd1da194fd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-config-volume\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971606 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971459 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971606 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c02dba-388a-47c1-ae01-38bd1da194fd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971606 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n459\" (UniqueName: \"kubernetes.io/projected/a8c02dba-388a-47c1-ae01-38bd1da194fd-kube-api-access-4n459\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971606 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-web-config\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.971606 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.971600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:02.972987 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.972969 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 11:15:02.983559 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:02.983529 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:15:03.072045 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072010 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-web-config\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072045 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8c02dba-388a-47c1-ae01-38bd1da194fd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8c02dba-388a-47c1-ae01-38bd1da194fd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8c02dba-388a-47c1-ae01-38bd1da194fd-config-out\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072207 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8c02dba-388a-47c1-ae01-38bd1da194fd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072243 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-config-volume\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c02dba-388a-47c1-ae01-38bd1da194fd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.072812 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072351 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n459\" (UniqueName: \"kubernetes.io/projected/a8c02dba-388a-47c1-ae01-38bd1da194fd-kube-api-access-4n459\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.073184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072908 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8c02dba-388a-47c1-ae01-38bd1da194fd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.073184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.072968 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a8c02dba-388a-47c1-ae01-38bd1da194fd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.073759 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.073732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8c02dba-388a-47c1-ae01-38bd1da194fd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.075505 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075475 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.075599 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075580 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.075665 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075610 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a8c02dba-388a-47c1-ae01-38bd1da194fd-config-out\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.075665 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075657 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-web-config\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.075773 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.075900 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075884 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.075969 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075951 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.076017 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.075999 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a8c02dba-388a-47c1-ae01-38bd1da194fd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.076904 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.076882 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a8c02dba-388a-47c1-ae01-38bd1da194fd-config-volume\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.082008 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.081990 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n459\" (UniqueName: \"kubernetes.io/projected/a8c02dba-388a-47c1-ae01-38bd1da194fd-kube-api-access-4n459\") pod \"alertmanager-main-0\" (UID: \"a8c02dba-388a-47c1-ae01-38bd1da194fd\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.271553 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.271513 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 11:15:03.565942 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.565916 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 11:15:03.567760 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:15:03.567729 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c02dba_388a_47c1_ae01_38bd1da194fd.slice/crio-2d49f28a09ef2a6ecf0e9acfffcf45f3e4479cef081f1a4138b5a355d090e4e1 WatchSource:0}: Error finding container 2d49f28a09ef2a6ecf0e9acfffcf45f3e4479cef081f1a4138b5a355d090e4e1: Status 404 returned error can't find the container with id 2d49f28a09ef2a6ecf0e9acfffcf45f3e4479cef081f1a4138b5a355d090e4e1 Sep 10 11:15:03.888273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.888177 2560 generic.go:356] "Generic (PLEG): container finished" podID="a8c02dba-388a-47c1-ae01-38bd1da194fd" containerID="563927f6f7625163730c3496d1aa5eec2d81cb40d73a1117e7e822abc6a12490" exitCode=0 Sep 10 11:15:03.888448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.888274 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerDied","Data":"563927f6f7625163730c3496d1aa5eec2d81cb40d73a1117e7e822abc6a12490"} Sep 10 11:15:03.888448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:03.888378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerStarted","Data":"2d49f28a09ef2a6ecf0e9acfffcf45f3e4479cef081f1a4138b5a355d090e4e1"} Sep 10 11:15:04.171808 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.171772 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccf8c940-fc81-4ccc-a4e0-a29d8dac0934" path="/var/lib/kubelet/pods/ccf8c940-fc81-4ccc-a4e0-a29d8dac0934/volumes" Sep 10 11:15:04.896849 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.896809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerStarted","Data":"cacb545d6e59be839bb465e2d57875ec9966f166f0778a5f9867c7cffbe33023"} Sep 10 11:15:04.897197 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.896874 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerStarted","Data":"f83309987346d75eed7f5b178358a70a7ef1b10143820ecf46111d5c05651653"} Sep 10 11:15:04.897197 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.896900 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerStarted","Data":"87bac7820d7851e3410dc9978b6a11f9f4ec2a4f1b241f443f0f264699fd4881"} Sep 10 11:15:04.897197 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.896924 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerStarted","Data":"3ea9056816f2b0b685c59576f53c8f158629fc522a2a2b9b5cf285b2431f79da"} Sep 10 11:15:04.897197 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.896948 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerStarted","Data":"e97fd29103b47acf629255e7f8326fe2c89b0a97901dd3f2fc6ece9f1a6cab83"} Sep 10 11:15:04.897197 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.896969 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a8c02dba-388a-47c1-ae01-38bd1da194fd","Type":"ContainerStarted","Data":"6dd841c723aa227137399de14c95dc38684093c4ac6bdd4fe0f7760a38aff3a2"} Sep 10 11:15:04.937322 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:04.933616 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.91305801 podStartE2EDuration="2.933593335s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.535325ms" isStatefulPod=false podCreationTimestamp="2026-09-10 11:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:04.9290856 +0000 UTC m=+143.288141811" watchObservedRunningTime="2026-09-10 11:15:04.933593335 +0000 UTC m=+143.292649540" Sep 10 11:15:05.278318 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.278263 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:15:05.278653 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.278605 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="prometheus" containerID="cri-o://898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530" gracePeriod=600 Sep 10 11:15:05.278840 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.278627 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy" containerID="cri-o://9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78" gracePeriod=600 Sep 10 11:15:05.278918 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.278659 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-thanos" containerID="cri-o://eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f" gracePeriod=600 Sep 10 11:15:05.278995 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.278659 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="thanos-sidecar" containerID="cri-o://66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74" gracePeriod=600 Sep 10 11:15:05.279086 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.278672 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="config-reloader" containerID="cri-o://ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c" gracePeriod=600 Sep 10 11:15:05.279837 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.278693 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-web" containerID="cri-o://276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70" gracePeriod=600 Sep 10 11:15:05.905652 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905618 2560 generic.go:356] "Generic (PLEG): container finished" podID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerID="eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f" exitCode=0 Sep 10 11:15:05.905652 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905642 2560 generic.go:356] "Generic (PLEG): container finished" podID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerID="9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78" exitCode=0 Sep 10 11:15:05.905652 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905649 2560 generic.go:356] "Generic (PLEG): container finished" podID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerID="66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74" exitCode=0 Sep 10 11:15:05.905652 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905656 2560 generic.go:356] "Generic (PLEG): container finished" podID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerID="ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c" exitCode=0 Sep 10 11:15:05.905652 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905662 2560 generic.go:356] "Generic (PLEG): container finished" podID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerID="898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530" exitCode=0 Sep 10 11:15:05.906165 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905685 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f"} Sep 10 11:15:05.906165 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905749 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78"} Sep 10 11:15:05.906165 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74"} Sep 10 11:15:05.906165 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905802 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c"} Sep 10 11:15:05.906165 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:05.905826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530"} Sep 10 11:15:06.530906 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.530882 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:06.606932 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.606894 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-kube-rbac-proxy\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.606932 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.606938 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-tls\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.606957 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.606980 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-thanos-prometheus-http-client-file\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607003 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-trusted-ca-bundle\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607026 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-tls-assets\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607054 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-db\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607079 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config-out\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607133 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-web-config\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607184 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607212 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-metrics-client-ca\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607236 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-grpc-tls\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607264 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-kubelet-serving-ca-bundle\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607295 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-2\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607339 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-metrics-client-certs\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607367 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-0\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607393 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-serving-certs-ca-bundle\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607422 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-1\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607449 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607476 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmcs9\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-kube-api-access-cmcs9\") pod \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\" (UID: \"79aa6da3-d13a-42ad-83e0-eb6e03b5e940\") " Sep 10 11:15:06.607596 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607498 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:06.608093 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607816 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.608093 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.607886 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:06.608351 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.608324 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:06.608464 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.608365 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 11:15:06.609193 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.608625 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:06.609193 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.608937 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:06.609193 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.609106 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:06.610945 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.610907 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:06.613442 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.613411 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.613876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.613831 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config" (OuterVolumeSpecName: "config") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.613968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.613867 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config-out" (OuterVolumeSpecName: "config-out") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 11:15:06.613968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.613868 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-kube-api-access-cmcs9" (OuterVolumeSpecName: "kube-api-access-cmcs9") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "kube-api-access-cmcs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:15:06.614067 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.613968 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.614239 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.614218 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:15:06.614382 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.614250 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.614382 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.614341 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.614495 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.614433 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.614495 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.614474 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.614598 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.614557 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.625533 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.625483 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-web-config" (OuterVolumeSpecName: "web-config") pod "79aa6da3-d13a-42ad-83e0-eb6e03b5e940" (UID: "79aa6da3-d13a-42ad-83e0-eb6e03b5e940"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:06.708869 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708835 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.708869 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708868 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-metrics-client-ca\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708878 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-grpc-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708887 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708896 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708904 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-metrics-client-certs\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708913 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708921 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708931 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708939 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708947 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmcs9\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-kube-api-access-cmcs9\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708955 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-kube-rbac-proxy\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708963 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708971 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708981 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708990 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-tls-assets\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.708998 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-prometheus-k8s-db\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.709006 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-config-out\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.709031 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.709013 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79aa6da3-d13a-42ad-83e0-eb6e03b5e940-web-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:06.913032 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.913000 2560 generic.go:356] "Generic (PLEG): container finished" podID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerID="276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70" exitCode=0 Sep 10 11:15:06.913461 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.913053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70"} Sep 10 11:15:06.913461 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.913077 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:06.913461 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.913111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"79aa6da3-d13a-42ad-83e0-eb6e03b5e940","Type":"ContainerDied","Data":"b077e2eeb1709fbc804c5ec3cd62ddcad01f5aeb911da9e5e98a1de0c3e11fa6"} Sep 10 11:15:06.913461 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.913143 2560 scope.go:117] "RemoveContainer" containerID="eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f" Sep 10 11:15:06.921835 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.921720 2560 scope.go:117] "RemoveContainer" containerID="9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78" Sep 10 11:15:06.929491 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.929471 2560 scope.go:117] "RemoveContainer" containerID="276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70" Sep 10 11:15:06.936669 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.936649 2560 scope.go:117] "RemoveContainer" containerID="66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74" Sep 10 11:15:06.942401 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.942376 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:15:06.944844 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.944825 2560 scope.go:117] "RemoveContainer" containerID="ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c" Sep 10 11:15:06.946147 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.946129 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:15:06.952276 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.952259 2560 scope.go:117] "RemoveContainer" containerID="898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530" Sep 10 11:15:06.960453 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.960432 2560 scope.go:117] "RemoveContainer" containerID="94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117" Sep 10 11:15:06.967612 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.967593 2560 scope.go:117] "RemoveContainer" containerID="eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f" Sep 10 11:15:06.967919 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:06.967899 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f\": container with ID starting with eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f not found: ID does not exist" containerID="eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f" Sep 10 11:15:06.967965 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.967927 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f"} err="failed to get container status \"eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f\": rpc error: code = NotFound desc = could not find container \"eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f\": container with ID starting with eec8965491c3fbeb37c4e67b0266f32d656ecd771f897a3caa095963274bde1f not found: ID does not exist" Sep 10 11:15:06.967965 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.967948 2560 scope.go:117] "RemoveContainer" containerID="9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78" Sep 10 11:15:06.968252 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:06.968234 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78\": container with ID starting with 9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78 not found: ID does not exist" containerID="9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78" Sep 10 11:15:06.968323 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.968258 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78"} err="failed to get container status \"9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78\": rpc error: code = NotFound desc = could not find container \"9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78\": container with ID starting with 9187099ee33935eec93d43c00ab68307d2cc82a28abf545c3328d740729fad78 not found: ID does not exist" Sep 10 11:15:06.968323 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.968274 2560 scope.go:117] "RemoveContainer" containerID="276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70" Sep 10 11:15:06.968512 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:06.968494 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70\": container with ID starting with 276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70 not found: ID does not exist" containerID="276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70" Sep 10 11:15:06.968545 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.968515 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70"} err="failed to get container status \"276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70\": rpc error: code = NotFound desc = could not find container \"276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70\": container with ID starting with 276de5e59c6c49bb4cd15a11b0b33dc25948961ab8406317648ff45a47b3aa70 not found: ID does not exist" Sep 10 11:15:06.968545 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.968527 2560 scope.go:117] "RemoveContainer" containerID="66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74" Sep 10 11:15:06.968753 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:06.968737 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74\": container with ID starting with 66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74 not found: ID does not exist" containerID="66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74" Sep 10 11:15:06.968799 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.968758 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74"} err="failed to get container status \"66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74\": rpc error: code = NotFound desc = could not find container \"66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74\": container with ID starting with 66f6142db4044e4d52aee0d25b9c3ee9c9032fdc1a5e651bf13ff2b9f65f9e74 not found: ID does not exist" Sep 10 11:15:06.968799 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.968773 2560 scope.go:117] "RemoveContainer" containerID="ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c" Sep 10 11:15:06.968994 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:06.968980 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c\": container with ID starting with ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c not found: ID does not exist" containerID="ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c" Sep 10 11:15:06.969038 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.968996 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c"} err="failed to get container status \"ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c\": rpc error: code = NotFound desc = could not find container \"ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c\": container with ID starting with ef88eebac675ce6d6924630225f6e899ab880267ac2e4f82b09327d606a9893c not found: ID does not exist" Sep 10 11:15:06.969038 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.969008 2560 scope.go:117] "RemoveContainer" containerID="898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530" Sep 10 11:15:06.969261 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:06.969245 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530\": container with ID starting with 898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530 not found: ID does not exist" containerID="898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530" Sep 10 11:15:06.969338 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.969262 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530"} err="failed to get container status \"898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530\": rpc error: code = NotFound desc = could not find container \"898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530\": container with ID starting with 898494aa30ebd5bb90563627202ec18a5fcd0ddedb82295d4d69cb0764703530 not found: ID does not exist" Sep 10 11:15:06.969338 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.969273 2560 scope.go:117] "RemoveContainer" containerID="94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117" Sep 10 11:15:06.969563 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:06.969546 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117\": container with ID starting with 94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117 not found: ID does not exist" containerID="94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117" Sep 10 11:15:06.969641 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.969566 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117"} err="failed to get container status \"94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117\": rpc error: code = NotFound desc = could not find container \"94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117\": container with ID starting with 94fa00330b54e4cbe748eee1baad2b3076bcde3ea4be29e245e870783b97d117 not found: ID does not exist" Sep 10 11:15:06.986063 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986032 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:15:06.986426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986412 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="thanos-sidecar" Sep 10 11:15:06.986474 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986428 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="thanos-sidecar" Sep 10 11:15:06.986474 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986450 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="config-reloader" Sep 10 11:15:06.986474 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986461 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="config-reloader" Sep 10 11:15:06.986474 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986469 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="init-config-reloader" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986475 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="init-config-reloader" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986482 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986488 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986495 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-thanos" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986502 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-thanos" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986510 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="prometheus" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986516 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="prometheus" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986522 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-web" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986528 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-web" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986582 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="thanos-sidecar" Sep 10 11:15:06.986585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986589 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-web" Sep 10 11:15:06.986876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986597 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy" Sep 10 11:15:06.986876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986602 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="kube-rbac-proxy-thanos" Sep 10 11:15:06.986876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986611 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="prometheus" Sep 10 11:15:06.986876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.986622 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" containerName="config-reloader" Sep 10 11:15:06.989682 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.989663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:06.998603 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.998569 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 11:15:06.998603 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.998584 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 11:15:06.998765 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.998636 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 11:15:06.998893 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.998880 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 11:15:06.999096 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.999081 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 11:15:06.999176 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.999162 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 11:15:06.999394 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.999374 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-31ju1bg6cghqd\"" Sep 10 11:15:06.999463 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.999395 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 11:15:06.999502 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:06.999484 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 11:15:07.000576 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.000559 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9tr75\"" Sep 10 11:15:07.003875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.003856 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 11:15:07.003875 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.003877 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 11:15:07.004344 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.004020 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 11:15:07.004344 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.004049 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 11:15:07.011934 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.011906 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf395214-0110-4f09-9feb-f90dd2fe02d1-config-out\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012085 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.011943 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012085 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.011976 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012164 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012118 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-config\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012207 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012171 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012252 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012206 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012252 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012238 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-web-config\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012342 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012267 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012387 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012387 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012476 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012476 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012453 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012545 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012517 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012583 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012546 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012630 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012579 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnvss\" (UniqueName: \"kubernetes.io/projected/bf395214-0110-4f09-9feb-f90dd2fe02d1-kube-api-access-dnvss\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012671 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 11:15:07.012771 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012683 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012771 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012712 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf395214-0110-4f09-9feb-f90dd2fe02d1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012771 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012736 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.012771 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.012759 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.014716 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.014698 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 11:15:07.030415 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.030382 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:15:07.114169 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114169 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf395214-0110-4f09-9feb-f90dd2fe02d1-config-out\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-config\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-web-config\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114482 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114511 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114607 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114641 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.114751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnvss\" (UniqueName: \"kubernetes.io/projected/bf395214-0110-4f09-9feb-f90dd2fe02d1-kube-api-access-dnvss\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.115245 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114777 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.115245 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114814 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf395214-0110-4f09-9feb-f90dd2fe02d1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.115245 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.114901 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.117830 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.117551 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.117830 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.117558 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.117830 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.117611 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.118074 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.117832 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.118174 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.118149 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.118243 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.118195 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119125 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.118812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119125 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.118840 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119125 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119068 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119125 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bf395214-0110-4f09-9feb-f90dd2fe02d1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119425 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119235 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119503 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119479 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bf395214-0110-4f09-9feb-f90dd2fe02d1-config-out\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119562 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119502 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bf395214-0110-4f09-9feb-f90dd2fe02d1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119615 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-config\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119671 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119645 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.119712 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.119666 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-web-config\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.120268 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.120251 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.120390 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.120375 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bf395214-0110-4f09-9feb-f90dd2fe02d1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.127092 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.127072 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnvss\" (UniqueName: \"kubernetes.io/projected/bf395214-0110-4f09-9feb-f90dd2fe02d1-kube-api-access-dnvss\") pod \"prometheus-k8s-0\" (UID: \"bf395214-0110-4f09-9feb-f90dd2fe02d1\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.299173 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.299078 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:07.603399 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.603035 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 11:15:07.608522 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:15:07.608420 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf395214_0110_4f09_9feb_f90dd2fe02d1.slice/crio-03129b210250919a96c27f6538736f32e2bc102a238f5ea1888b0bc71366176f WatchSource:0}: Error finding container 03129b210250919a96c27f6538736f32e2bc102a238f5ea1888b0bc71366176f: Status 404 returned error can't find the container with id 03129b210250919a96c27f6538736f32e2bc102a238f5ea1888b0bc71366176f Sep 10 11:15:07.919769 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.919735 2560 generic.go:356] "Generic (PLEG): container finished" podID="bf395214-0110-4f09-9feb-f90dd2fe02d1" containerID="34164cef0b5def63eaf2c630d6ec56a7b4c136ae7923b40c875c4d2bc2a76d8f" exitCode=0 Sep 10 11:15:07.920128 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.919788 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerDied","Data":"34164cef0b5def63eaf2c630d6ec56a7b4c136ae7923b40c875c4d2bc2a76d8f"} Sep 10 11:15:07.920128 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:07.919838 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerStarted","Data":"03129b210250919a96c27f6538736f32e2bc102a238f5ea1888b0bc71366176f"} Sep 10 11:15:08.176406 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.176367 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79aa6da3-d13a-42ad-83e0-eb6e03b5e940" path="/var/lib/kubelet/pods/79aa6da3-d13a-42ad-83e0-eb6e03b5e940/volumes" Sep 10 11:15:08.262435 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.262408 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86c4f99ffd-n7pq5"] Sep 10 11:15:08.265275 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.265250 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.280817 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.280795 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c4f99ffd-n7pq5"] Sep 10 11:15:08.327420 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.327380 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-trusted-ca-bundle\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.327420 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.327423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-service-ca\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.327641 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.327495 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-console-config\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.327641 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.327548 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-serving-cert\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.327641 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.327566 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-oauth-config\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.327641 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.327627 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-oauth-serving-cert\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.327822 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.327671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj45p\" (UniqueName: \"kubernetes.io/projected/82ed813e-5ded-47de-b72e-d96b83300f4f-kube-api-access-mj45p\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.337629 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.337603 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:08.415274 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.415243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:08.428454 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.428394 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj45p\" (UniqueName: \"kubernetes.io/projected/82ed813e-5ded-47de-b72e-d96b83300f4f-kube-api-access-mj45p\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.428541 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.428460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-trusted-ca-bundle\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.428541 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.428485 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-service-ca\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.428541 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.428512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-console-config\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.428735 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.428546 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-serving-cert\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.428735 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.428574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-oauth-config\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.428735 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.428617 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-oauth-serving-cert\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.429335 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.429285 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-console-config\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.429430 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.429404 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-service-ca\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.429487 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.429454 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-oauth-serving-cert\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.429581 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.429566 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-trusted-ca-bundle\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.431604 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.431582 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-serving-cert\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.431684 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.431658 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-oauth-config\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.437710 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.437686 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj45p\" (UniqueName: \"kubernetes.io/projected/82ed813e-5ded-47de-b72e-d96b83300f4f-kube-api-access-mj45p\") pod \"console-86c4f99ffd-n7pq5\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.577580 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.577539 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:08.592615 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.592591 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:08.672695 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.672661 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:08.857603 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.857282 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c4f99ffd-n7pq5"] Sep 10 11:15:08.859841 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:15:08.859815 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ed813e_5ded_47de_b72e_d96b83300f4f.slice/crio-b9b46f7f46d91ce5d74b65991370b7bb005b85ba0222bc9bca098d1d1f07c59b WatchSource:0}: Error finding container b9b46f7f46d91ce5d74b65991370b7bb005b85ba0222bc9bca098d1d1f07c59b: Status 404 returned error can't find the container with id b9b46f7f46d91ce5d74b65991370b7bb005b85ba0222bc9bca098d1d1f07c59b Sep 10 11:15:08.925593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.925560 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c4f99ffd-n7pq5" event={"ID":"82ed813e-5ded-47de-b72e-d96b83300f4f","Type":"ContainerStarted","Data":"8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16"} Sep 10 11:15:08.925980 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.925617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c4f99ffd-n7pq5" event={"ID":"82ed813e-5ded-47de-b72e-d96b83300f4f","Type":"ContainerStarted","Data":"b9b46f7f46d91ce5d74b65991370b7bb005b85ba0222bc9bca098d1d1f07c59b"} Sep 10 11:15:08.929267 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.929240 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerStarted","Data":"0c93d7ade3e05251c52acaf61fb6d55db518e0bddf7ae840945fa98e70706263"} Sep 10 11:15:08.929390 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.929311 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerStarted","Data":"36840ca50d5b02cd69c0fe225b83ed88726c765d6b1d3a0d8ac321388797b280"} Sep 10 11:15:08.929390 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.929348 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerStarted","Data":"a6fee9041f5417414d6055e1e64b88242c3fe39e10b19d41ceec65cc5423b89c"} Sep 10 11:15:08.929390 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.929373 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerStarted","Data":"0d8cdf9a8b7c75012a0b8e937387c1c6401af892e8b7458c1d124d4bffbf63d3"} Sep 10 11:15:08.929540 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.929408 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerStarted","Data":"1ab132c07b484cbd94240ac740edd29ec78942cc0fcd24c33e0dc631d58dcabe"} Sep 10 11:15:08.929540 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.929435 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bf395214-0110-4f09-9feb-f90dd2fe02d1","Type":"ContainerStarted","Data":"f7a20004fc79ebde05e6ca680d8c0c66c8150fb784605b469ff2d3a8a5f16033"} Sep 10 11:15:08.949860 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.949794 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86c4f99ffd-n7pq5" podStartSLOduration=0.949778767 podStartE2EDuration="949.778767ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:08.948795153 +0000 UTC m=+147.307851361" watchObservedRunningTime="2026-09-10 11:15:08.949778767 +0000 UTC m=+147.308834972" Sep 10 11:15:08.989932 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:08.989878 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.972308694 podStartE2EDuration="2.98986272s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.554026ms" isStatefulPod=false podCreationTimestamp="2026-09-10 11:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:08.985208791 +0000 UTC m=+147.344264996" watchObservedRunningTime="2026-09-10 11:15:08.98986272 +0000 UTC m=+147.348918925" Sep 10 11:15:12.299397 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:12.299359 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:15:12.893273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:12.893238 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:12.990607 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:12.990574 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:18.577859 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:18.577818 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:18.578240 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:18.577884 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:18.582519 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:18.582495 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:18.978971 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:18.978941 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:15:19.039799 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:19.039770 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5449b7878f-lklsl"] Sep 10 11:15:19.179639 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:19.179604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:19.281441 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:19.281358 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:19.357457 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:19.357422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:36.878728 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:36.878692 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:15:44.066219 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.066155 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5449b7878f-lklsl" podUID="1d5972ea-3aef-48a5-9b86-4ecf50839f7a" containerName="console" containerID="cri-o://0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474" gracePeriod=15 Sep 10 11:15:44.310008 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.309980 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:15:44.356489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356406 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjxbz\" (UniqueName: \"kubernetes.io/projected/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-kube-api-access-gjxbz\") pod \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " Sep 10 11:15:44.356489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356463 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-service-ca\") pod \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " Sep 10 11:15:44.356717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356504 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-config\") pod \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " Sep 10 11:15:44.356717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356528 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-oauth-serving-cert\") pod \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " Sep 10 11:15:44.356717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356549 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-trusted-ca-bundle\") pod \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " Sep 10 11:15:44.356717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356607 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-oauth-config\") pod \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " Sep 10 11:15:44.356717 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356644 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-serving-cert\") pod \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\" (UID: \"1d5972ea-3aef-48a5-9b86-4ecf50839f7a\") " Sep 10 11:15:44.356966 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356933 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1d5972ea-3aef-48a5-9b86-4ecf50839f7a" (UID: "1d5972ea-3aef-48a5-9b86-4ecf50839f7a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:44.356966 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356951 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-service-ca" (OuterVolumeSpecName: "service-ca") pod "1d5972ea-3aef-48a5-9b86-4ecf50839f7a" (UID: "1d5972ea-3aef-48a5-9b86-4ecf50839f7a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:44.357074 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.356940 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-config" (OuterVolumeSpecName: "console-config") pod "1d5972ea-3aef-48a5-9b86-4ecf50839f7a" (UID: "1d5972ea-3aef-48a5-9b86-4ecf50839f7a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:44.357074 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.357024 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1d5972ea-3aef-48a5-9b86-4ecf50839f7a" (UID: "1d5972ea-3aef-48a5-9b86-4ecf50839f7a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:15:44.358705 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.358682 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-kube-api-access-gjxbz" (OuterVolumeSpecName: "kube-api-access-gjxbz") pod "1d5972ea-3aef-48a5-9b86-4ecf50839f7a" (UID: "1d5972ea-3aef-48a5-9b86-4ecf50839f7a"). InnerVolumeSpecName "kube-api-access-gjxbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:15:44.358811 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.358776 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1d5972ea-3aef-48a5-9b86-4ecf50839f7a" (UID: "1d5972ea-3aef-48a5-9b86-4ecf50839f7a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:44.358935 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.358911 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1d5972ea-3aef-48a5-9b86-4ecf50839f7a" (UID: "1d5972ea-3aef-48a5-9b86-4ecf50839f7a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:15:44.458346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.458288 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:44.458346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.458345 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-oauth-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:44.458346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.458360 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-trusted-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:44.458588 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.458371 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-oauth-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:44.458588 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.458384 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-console-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:44.458588 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.458396 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjxbz\" (UniqueName: \"kubernetes.io/projected/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-kube-api-access-gjxbz\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:44.458588 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:44.458408 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d5972ea-3aef-48a5-9b86-4ecf50839f7a-service-ca\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:15:45.080635 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.080600 2560 generic.go:356] "Generic (PLEG): container finished" podID="1d5972ea-3aef-48a5-9b86-4ecf50839f7a" containerID="0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474" exitCode=0 Sep 10 11:15:45.081185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.080652 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5449b7878f-lklsl" Sep 10 11:15:45.081185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.080653 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5449b7878f-lklsl" event={"ID":"1d5972ea-3aef-48a5-9b86-4ecf50839f7a","Type":"ContainerDied","Data":"0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474"} Sep 10 11:15:45.081185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.080782 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5449b7878f-lklsl" event={"ID":"1d5972ea-3aef-48a5-9b86-4ecf50839f7a","Type":"ContainerDied","Data":"52191fe5a4a6ff2a933208b895e36ecb43916deedbbf403dc8ba6a468a0cb995"} Sep 10 11:15:45.081185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.080813 2560 scope.go:117] "RemoveContainer" containerID="0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474" Sep 10 11:15:45.089529 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.089505 2560 scope.go:117] "RemoveContainer" containerID="0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474" Sep 10 11:15:45.089802 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:15:45.089783 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474\": container with ID starting with 0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474 not found: ID does not exist" containerID="0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474" Sep 10 11:15:45.089843 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.089813 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474"} err="failed to get container status \"0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474\": rpc error: code = NotFound desc = could not find container \"0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474\": container with ID starting with 0e588bcdc0efb961a3d65b3f67194f98d267d53446a8e7bfd9daa84e64043474 not found: ID does not exist" Sep 10 11:15:45.104590 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.104559 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5449b7878f-lklsl"] Sep 10 11:15:45.108004 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:45.107978 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5449b7878f-lklsl"] Sep 10 11:15:46.176184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:15:46.176146 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d5972ea-3aef-48a5-9b86-4ecf50839f7a" path="/var/lib/kubelet/pods/1d5972ea-3aef-48a5-9b86-4ecf50839f7a/volumes" Sep 10 11:16:02.627762 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:02.627730 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:16:02.704920 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:02.704888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:16:07.299466 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:07.299429 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:16:07.314937 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:07.314912 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:16:08.187154 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:08.187127 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 11:16:50.902168 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:50.902112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:16:53.406503 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:53.406467 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:16:53.468849 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:16:53.468816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:20.304425 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:20.304336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:22.132953 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:22.132919 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:42.048811 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:42.048777 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:17:42.051691 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:42.051668 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:17:42.053416 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:42.053397 2560 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 11:17:54.990177 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:54.990141 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:55.057451 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:55.057416 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:56.850471 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:56.850436 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:56.918070 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:56.918038 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:57.011059 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:57.011020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:57.107896 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:57.107810 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:57.601936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:57.601899 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:57.707946 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:57.707890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:57.806765 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:57.806728 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:57.903795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:57.903696 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:58.005962 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:58.005925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:17:58.238608 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:17:58.238562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:43.125840 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.125809 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6699dbc47b-gf8xr"] Sep 10 11:18:43.126256 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.126156 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d5972ea-3aef-48a5-9b86-4ecf50839f7a" containerName="console" Sep 10 11:18:43.126256 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.126167 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d5972ea-3aef-48a5-9b86-4ecf50839f7a" containerName="console" Sep 10 11:18:43.126256 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.126232 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d5972ea-3aef-48a5-9b86-4ecf50839f7a" containerName="console" Sep 10 11:18:43.129345 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.129323 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.152146 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.152113 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6699dbc47b-gf8xr"] Sep 10 11:18:43.212971 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.212915 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:43.319758 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.319717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-config\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.319758 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.319758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-oauth-serving-cert\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.319973 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.319784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m6mb\" (UniqueName: \"kubernetes.io/projected/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-kube-api-access-9m6mb\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.319973 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.319873 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-serving-cert\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.319973 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.319905 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-oauth-config\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.319973 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.319921 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-trusted-ca-bundle\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.319973 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.319966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-service-ca\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.332222 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.332188 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:43.421265 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.421231 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-serving-cert\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.421438 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.421270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-oauth-config\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.421438 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.421286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-trusted-ca-bundle\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.421438 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.421361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-service-ca\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.421438 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.421431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-config\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.421626 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.421450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-oauth-serving-cert\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.421626 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.421491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6mb\" (UniqueName: \"kubernetes.io/projected/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-kube-api-access-9m6mb\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.422163 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.422137 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-service-ca\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.422265 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.422165 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-config\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.422334 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.422283 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-oauth-serving-cert\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.422426 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.422409 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-trusted-ca-bundle\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.424374 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.424343 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-oauth-config\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.424459 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.424421 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-console-serving-cert\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.426264 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.426242 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:43.430660 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.430630 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m6mb\" (UniqueName: \"kubernetes.io/projected/d211812c-aaaa-4c9e-aca3-c3959f5a9bb7-kube-api-access-9m6mb\") pod \"console-6699dbc47b-gf8xr\" (UID: \"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7\") " pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.439369 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.439343 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:43.724731 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.724577 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6699dbc47b-gf8xr"] Sep 10 11:18:43.727575 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:18:43.727544 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd211812c_aaaa_4c9e_aca3_c3959f5a9bb7.slice/crio-a08d1d7ddae1d4d988247ea017cbf098f7fd77c622bf158f339cb45a258a196b WatchSource:0}: Error finding container a08d1d7ddae1d4d988247ea017cbf098f7fd77c622bf158f339cb45a258a196b: Status 404 returned error can't find the container with id a08d1d7ddae1d4d988247ea017cbf098f7fd77c622bf158f339cb45a258a196b Sep 10 11:18:43.782654 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.782618 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6699dbc47b-gf8xr" event={"ID":"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7","Type":"ContainerStarted","Data":"a08d1d7ddae1d4d988247ea017cbf098f7fd77c622bf158f339cb45a258a196b"} Sep 10 11:18:43.924314 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:43.924279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:44.788888 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:44.788848 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6699dbc47b-gf8xr" event={"ID":"d211812c-aaaa-4c9e-aca3-c3959f5a9bb7","Type":"ContainerStarted","Data":"d50f30c680882e3137cfaa8bd0fda4af952809e646fd2ceced681a5713696fa2"} Sep 10 11:18:44.810450 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:44.810359 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6699dbc47b-gf8xr" podStartSLOduration=1.810338236 podStartE2EDuration="1.810338236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:18:44.809566508 +0000 UTC m=+363.168622716" watchObservedRunningTime="2026-09-10 11:18:44.810338236 +0000 UTC m=+363.169394445" Sep 10 11:18:46.958495 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:46.958457 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:48.091448 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:48.091408 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:53.440238 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:53.440192 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:53.440773 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:53.440263 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:53.445039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:53.445013 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:53.827710 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:53.827620 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6699dbc47b-gf8xr" Sep 10 11:18:53.877167 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:53.877133 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c4f99ffd-n7pq5"] Sep 10 11:18:54.000432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:54.000392 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:54.103078 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:54.102983 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:18:54.168457 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:18:54.168422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:18.897097 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:18.897037 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86c4f99ffd-n7pq5" podUID="82ed813e-5ded-47de-b72e-d96b83300f4f" containerName="console" containerID="cri-o://8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16" gracePeriod=15 Sep 10 11:19:18.975839 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:18.975797 2560 patch_prober.go:28] interesting pod/console-86c4f99ffd-n7pq5 container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" start-of-body= Sep 10 11:19:18.976000 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:18.975873 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-86c4f99ffd-n7pq5" podUID="82ed813e-5ded-47de-b72e-d96b83300f4f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" Sep 10 11:19:19.136282 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.136255 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:19:19.242119 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242086 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-service-ca\") pod \"82ed813e-5ded-47de-b72e-d96b83300f4f\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " Sep 10 11:19:19.242292 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242148 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj45p\" (UniqueName: \"kubernetes.io/projected/82ed813e-5ded-47de-b72e-d96b83300f4f-kube-api-access-mj45p\") pod \"82ed813e-5ded-47de-b72e-d96b83300f4f\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " Sep 10 11:19:19.242292 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242246 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-oauth-serving-cert\") pod \"82ed813e-5ded-47de-b72e-d96b83300f4f\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " Sep 10 11:19:19.242292 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242270 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-oauth-config\") pod \"82ed813e-5ded-47de-b72e-d96b83300f4f\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " Sep 10 11:19:19.242490 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242319 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-serving-cert\") pod \"82ed813e-5ded-47de-b72e-d96b83300f4f\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " Sep 10 11:19:19.242490 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242340 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-console-config\") pod \"82ed813e-5ded-47de-b72e-d96b83300f4f\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " Sep 10 11:19:19.242490 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242378 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-trusted-ca-bundle\") pod \"82ed813e-5ded-47de-b72e-d96b83300f4f\" (UID: \"82ed813e-5ded-47de-b72e-d96b83300f4f\") " Sep 10 11:19:19.242641 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242612 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "82ed813e-5ded-47de-b72e-d96b83300f4f" (UID: "82ed813e-5ded-47de-b72e-d96b83300f4f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:19:19.242704 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242639 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-service-ca" (OuterVolumeSpecName: "service-ca") pod "82ed813e-5ded-47de-b72e-d96b83300f4f" (UID: "82ed813e-5ded-47de-b72e-d96b83300f4f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:19:19.242766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242716 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-console-config" (OuterVolumeSpecName: "console-config") pod "82ed813e-5ded-47de-b72e-d96b83300f4f" (UID: "82ed813e-5ded-47de-b72e-d96b83300f4f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:19:19.242965 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.242935 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "82ed813e-5ded-47de-b72e-d96b83300f4f" (UID: "82ed813e-5ded-47de-b72e-d96b83300f4f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:19:19.244352 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.244325 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ed813e-5ded-47de-b72e-d96b83300f4f-kube-api-access-mj45p" (OuterVolumeSpecName: "kube-api-access-mj45p") pod "82ed813e-5ded-47de-b72e-d96b83300f4f" (UID: "82ed813e-5ded-47de-b72e-d96b83300f4f"). InnerVolumeSpecName "kube-api-access-mj45p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:19:19.244446 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.244379 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "82ed813e-5ded-47de-b72e-d96b83300f4f" (UID: "82ed813e-5ded-47de-b72e-d96b83300f4f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:19:19.244446 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.244400 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "82ed813e-5ded-47de-b72e-d96b83300f4f" (UID: "82ed813e-5ded-47de-b72e-d96b83300f4f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:19:19.343628 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.343599 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj45p\" (UniqueName: \"kubernetes.io/projected/82ed813e-5ded-47de-b72e-d96b83300f4f-kube-api-access-mj45p\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:19:19.343628 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.343624 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-oauth-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:19:19.343628 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.343634 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-oauth-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:19:19.343851 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.343643 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82ed813e-5ded-47de-b72e-d96b83300f4f-console-serving-cert\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:19:19.343851 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.343655 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-console-config\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:19:19.343851 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.343663 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-trusted-ca-bundle\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:19:19.343851 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.343672 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82ed813e-5ded-47de-b72e-d96b83300f4f-service-ca\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:19:19.930774 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.930742 2560 generic.go:356] "Generic (PLEG): container finished" podID="82ed813e-5ded-47de-b72e-d96b83300f4f" containerID="8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16" exitCode=0 Sep 10 11:19:19.931181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.930786 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c4f99ffd-n7pq5" Sep 10 11:19:19.931181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.930809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c4f99ffd-n7pq5" event={"ID":"82ed813e-5ded-47de-b72e-d96b83300f4f","Type":"ContainerDied","Data":"8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16"} Sep 10 11:19:19.931181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.930847 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c4f99ffd-n7pq5" event={"ID":"82ed813e-5ded-47de-b72e-d96b83300f4f","Type":"ContainerDied","Data":"b9b46f7f46d91ce5d74b65991370b7bb005b85ba0222bc9bca098d1d1f07c59b"} Sep 10 11:19:19.931181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.930868 2560 scope.go:117] "RemoveContainer" containerID="8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16" Sep 10 11:19:19.939472 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.939456 2560 scope.go:117] "RemoveContainer" containerID="8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16" Sep 10 11:19:19.939747 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:19:19.939731 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16\": container with ID starting with 8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16 not found: ID does not exist" containerID="8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16" Sep 10 11:19:19.939790 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.939755 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16"} err="failed to get container status \"8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16\": rpc error: code = NotFound desc = could not find container \"8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16\": container with ID starting with 8607481a86df90855ba50dd3591297fc14b5ed05f20f01d23065d533edfb7a16 not found: ID does not exist" Sep 10 11:19:19.952422 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.952397 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c4f99ffd-n7pq5"] Sep 10 11:19:19.957904 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:19.957866 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86c4f99ffd-n7pq5"] Sep 10 11:19:20.170739 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:20.170708 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ed813e-5ded-47de-b72e-d96b83300f4f" path="/var/lib/kubelet/pods/82ed813e-5ded-47de-b72e-d96b83300f4f/volumes" Sep 10 11:19:50.069157 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.069124 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:50.155621 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.155583 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:50.318584 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.318550 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:50.422573 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.422539 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:50.525102 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.525070 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:50.695654 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.695557 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:50.848815 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.848781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:19:50.935237 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:19:50.935204 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:20:13.423219 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:20:13.423127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:20:59.560097 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:20:59.560050 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:06.539657 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:06.539624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:14.958540 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:14.958504 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:15.058430 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:15.058393 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:15.406804 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:15.406732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:15.515542 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:15.515505 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:15.628050 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:15.628013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:15.753886 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:15.753856 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:15.837988 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:15.837952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:17.548539 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:17.548510 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:17.621777 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:17.621736 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:21:55.786878 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:21:55.786842 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:22:42.078284 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:22:42.078256 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:22:42.078738 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:22:42.078676 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:22:57.770604 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:22:57.770571 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:22:59.457475 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:22:59.457432 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:03.069884 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:03.069843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:40.814316 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:40.814272 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:41.064128 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.064095 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-j2zwc"] Sep 10 11:23:41.064567 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.064517 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ed813e-5ded-47de-b72e-d96b83300f4f" containerName="console" Sep 10 11:23:41.064567 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.064536 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ed813e-5ded-47de-b72e-d96b83300f4f" containerName="console" Sep 10 11:23:41.064707 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.064649 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ed813e-5ded-47de-b72e-d96b83300f4f" containerName="console" Sep 10 11:23:41.067611 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.067590 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-j2zwc" Sep 10 11:23:41.070080 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.070059 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 11:23:41.070199 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.070089 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 11:23:41.070965 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.070944 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-xfghn\"" Sep 10 11:23:41.072539 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.072512 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:41.074213 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.074193 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-j2zwc"] Sep 10 11:23:41.134169 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.134134 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:41.157430 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.157396 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwvdr\" (UniqueName: \"kubernetes.io/projected/87dae763-33ee-4873-bb40-1ce5aec0779d-kube-api-access-kwvdr\") pod \"redis-69bbd798b6-j2zwc\" (UID: \"87dae763-33ee-4873-bb40-1ce5aec0779d\") " pod="test-ns-feast/redis-69bbd798b6-j2zwc" Sep 10 11:23:41.258458 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.258423 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwvdr\" (UniqueName: \"kubernetes.io/projected/87dae763-33ee-4873-bb40-1ce5aec0779d-kube-api-access-kwvdr\") pod \"redis-69bbd798b6-j2zwc\" (UID: \"87dae763-33ee-4873-bb40-1ce5aec0779d\") " pod="test-ns-feast/redis-69bbd798b6-j2zwc" Sep 10 11:23:41.266782 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.266755 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwvdr\" (UniqueName: \"kubernetes.io/projected/87dae763-33ee-4873-bb40-1ce5aec0779d-kube-api-access-kwvdr\") pod \"redis-69bbd798b6-j2zwc\" (UID: \"87dae763-33ee-4873-bb40-1ce5aec0779d\") " pod="test-ns-feast/redis-69bbd798b6-j2zwc" Sep 10 11:23:41.386004 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.385916 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-j2zwc" Sep 10 11:23:41.653869 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.653788 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-j2zwc"] Sep 10 11:23:41.656979 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:23:41.656953 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87dae763_33ee_4873_bb40_1ce5aec0779d.slice/crio-ba3d85e4b3a1401413aaa70329836e6c81033f5910fd419b2884b90b1b819432 WatchSource:0}: Error finding container ba3d85e4b3a1401413aaa70329836e6c81033f5910fd419b2884b90b1b819432: Status 404 returned error can't find the container with id ba3d85e4b3a1401413aaa70329836e6c81033f5910fd419b2884b90b1b819432 Sep 10 11:23:41.658860 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:41.658841 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 11:23:42.014358 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:42.014323 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-j2zwc" event={"ID":"87dae763-33ee-4873-bb40-1ce5aec0779d","Type":"ContainerStarted","Data":"ba3d85e4b3a1401413aaa70329836e6c81033f5910fd419b2884b90b1b819432"} Sep 10 11:23:44.025181 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:44.025140 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-j2zwc" event={"ID":"87dae763-33ee-4873-bb40-1ce5aec0779d","Type":"ContainerStarted","Data":"64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022"} Sep 10 11:23:44.041218 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:44.041158 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-j2zwc" podStartSLOduration=1.388528365 podStartE2EDuration="3.04113884s" totalImagesPullingTime="1.652610475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:23:44.039722326 +0000 UTC m=+662.398778544" watchObservedRunningTime="2026-09-10 11:23:44.04113884 +0000 UTC m=+662.400195045" Sep 10 11:23:49.798468 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:49.798424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:49.880575 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:49.880539 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:49.955508 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:49.955469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:50.114411 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:50.114314 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:23:50.214858 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:23:50.214821 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:24:15.220732 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:15.220695 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:24:16.599477 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.599442 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-2mvhh"] Sep 10 11:24:16.607356 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.607326 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2mvhh" Sep 10 11:24:16.610827 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.610776 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 10 11:24:16.610985 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.610840 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-56l9w\"" Sep 10 11:24:16.613727 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.613705 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-2mvhh"] Sep 10 11:24:16.672912 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.672868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdpwg\" (UniqueName: \"kubernetes.io/projected/7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05-kube-api-access-cdpwg\") pod \"feast-test-apply-2mvhh\" (UID: \"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05\") " pod="test-ns-feast/feast-test-apply-2mvhh" Sep 10 11:24:16.774290 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.774254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdpwg\" (UniqueName: \"kubernetes.io/projected/7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05-kube-api-access-cdpwg\") pod \"feast-test-apply-2mvhh\" (UID: \"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05\") " pod="test-ns-feast/feast-test-apply-2mvhh" Sep 10 11:24:16.783553 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.783519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdpwg\" (UniqueName: \"kubernetes.io/projected/7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05-kube-api-access-cdpwg\") pod \"feast-test-apply-2mvhh\" (UID: \"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05\") " pod="test-ns-feast/feast-test-apply-2mvhh" Sep 10 11:24:16.916843 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:16.916814 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2mvhh" Sep 10 11:24:17.191582 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:17.191547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-2mvhh"] Sep 10 11:24:17.193666 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:24:17.193641 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fdbabeb_9a06_4b51_acb7_a7baaaeb5a05.slice/crio-ae0be16583f5bab6194eea80942dfbdcccfe101dae0dca080bf590c5d187b66f WatchSource:0}: Error finding container ae0be16583f5bab6194eea80942dfbdcccfe101dae0dca080bf590c5d187b66f: Status 404 returned error can't find the container with id ae0be16583f5bab6194eea80942dfbdcccfe101dae0dca080bf590c5d187b66f Sep 10 11:24:18.170711 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:18.170675 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2mvhh" event={"ID":"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05","Type":"ContainerStarted","Data":"ae0be16583f5bab6194eea80942dfbdcccfe101dae0dca080bf590c5d187b66f"} Sep 10 11:24:23.192376 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:23.192333 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2mvhh" event={"ID":"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05","Type":"ContainerStarted","Data":"c65ccc119d9fb7bad2fad5f6a8d944576f1cafaee5a41ce8a61b5dbb33dd468d"} Sep 10 11:24:30.220988 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:30.220950 2560 generic.go:356] "Generic (PLEG): container finished" podID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" containerID="c65ccc119d9fb7bad2fad5f6a8d944576f1cafaee5a41ce8a61b5dbb33dd468d" exitCode=0 Sep 10 11:24:30.221386 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:30.221022 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2mvhh" event={"ID":"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05","Type":"ContainerDied","Data":"c65ccc119d9fb7bad2fad5f6a8d944576f1cafaee5a41ce8a61b5dbb33dd468d"} Sep 10 11:24:31.227097 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:31.227064 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2mvhh" event={"ID":"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05","Type":"ContainerStarted","Data":"224a1de2439687c83d3bdf3fa916cc69e5bacb2a12ef7a462ddb6574c6d4c244"} Sep 10 11:24:31.245241 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:31.245195 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-2mvhh" podStartSLOduration=3.3020953410000002 podStartE2EDuration="15.245180055s" totalImagesPullingTime="5.551550397s" totalInitContainerRuntime="6.391534317s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:24:31.24436781 +0000 UTC m=+709.603424018" watchObservedRunningTime="2026-09-10 11:24:31.245180055 +0000 UTC m=+709.604236260" Sep 10 11:24:40.265285 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:40.265248 2560 generic.go:356] "Generic (PLEG): container finished" podID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" containerID="224a1de2439687c83d3bdf3fa916cc69e5bacb2a12ef7a462ddb6574c6d4c244" exitCode=0 Sep 10 11:24:40.265722 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:40.265321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2mvhh" event={"ID":"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05","Type":"ContainerDied","Data":"224a1de2439687c83d3bdf3fa916cc69e5bacb2a12ef7a462ddb6574c6d4c244"} Sep 10 11:24:41.409312 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:41.409273 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2mvhh" Sep 10 11:24:41.600265 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:41.600170 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdpwg\" (UniqueName: \"kubernetes.io/projected/7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05-kube-api-access-cdpwg\") pod \"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05\" (UID: \"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05\") " Sep 10 11:24:41.602320 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:41.602286 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05-kube-api-access-cdpwg" (OuterVolumeSpecName: "kube-api-access-cdpwg") pod "7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" (UID: "7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05"). InnerVolumeSpecName "kube-api-access-cdpwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:24:41.701636 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:41.701596 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdpwg\" (UniqueName: \"kubernetes.io/projected/7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05-kube-api-access-cdpwg\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:24:42.276393 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:42.276344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2mvhh" event={"ID":"7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05","Type":"ContainerDied","Data":"ae0be16583f5bab6194eea80942dfbdcccfe101dae0dca080bf590c5d187b66f"} Sep 10 11:24:42.276393 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:42.276380 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2mvhh" Sep 10 11:24:42.276393 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:42.276395 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae0be16583f5bab6194eea80942dfbdcccfe101dae0dca080bf590c5d187b66f" Sep 10 11:24:42.445392 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:42.445268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-2mvhh_7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05/feast-1/0.log" Sep 10 11:24:42.451693 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:24:42.451669 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-2mvhh_7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05/feast-0/0.log" Sep 10 11:25:13.860715 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:13.860678 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:25:18.840586 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:18.840520 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-2mvhh"] Sep 10 11:25:18.845446 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:18.845416 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-2mvhh"] Sep 10 11:25:18.858791 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:18.858751 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-j2zwc"] Sep 10 11:25:18.858961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:18.858927 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-j2zwc" podUID="87dae763-33ee-4873-bb40-1ce5aec0779d" containerName="redis" containerID="cri-o://64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022" gracePeriod=30 Sep 10 11:25:18.913340 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:18.913292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:25:19.095333 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.095248 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-j2zwc" Sep 10 11:25:19.240589 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.240545 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwvdr\" (UniqueName: \"kubernetes.io/projected/87dae763-33ee-4873-bb40-1ce5aec0779d-kube-api-access-kwvdr\") pod \"87dae763-33ee-4873-bb40-1ce5aec0779d\" (UID: \"87dae763-33ee-4873-bb40-1ce5aec0779d\") " Sep 10 11:25:19.242590 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.242557 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87dae763-33ee-4873-bb40-1ce5aec0779d-kube-api-access-kwvdr" (OuterVolumeSpecName: "kube-api-access-kwvdr") pod "87dae763-33ee-4873-bb40-1ce5aec0779d" (UID: "87dae763-33ee-4873-bb40-1ce5aec0779d"). InnerVolumeSpecName "kube-api-access-kwvdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:25:19.341184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.341147 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwvdr\" (UniqueName: \"kubernetes.io/projected/87dae763-33ee-4873-bb40-1ce5aec0779d-kube-api-access-kwvdr\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:25:19.436077 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.436038 2560 generic.go:356] "Generic (PLEG): container finished" podID="87dae763-33ee-4873-bb40-1ce5aec0779d" containerID="64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022" exitCode=0 Sep 10 11:25:19.436278 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.436083 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-j2zwc" Sep 10 11:25:19.436278 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.436124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-j2zwc" event={"ID":"87dae763-33ee-4873-bb40-1ce5aec0779d","Type":"ContainerDied","Data":"64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022"} Sep 10 11:25:19.436278 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.436171 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-j2zwc" event={"ID":"87dae763-33ee-4873-bb40-1ce5aec0779d","Type":"ContainerDied","Data":"ba3d85e4b3a1401413aaa70329836e6c81033f5910fd419b2884b90b1b819432"} Sep 10 11:25:19.436278 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.436190 2560 scope.go:117] "RemoveContainer" containerID="64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022" Sep 10 11:25:19.451570 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.451548 2560 scope.go:117] "RemoveContainer" containerID="64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022" Sep 10 11:25:19.451885 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:25:19.451863 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022\": container with ID starting with 64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022 not found: ID does not exist" containerID="64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022" Sep 10 11:25:19.451955 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.451892 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022"} err="failed to get container status \"64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022\": rpc error: code = NotFound desc = could not find container \"64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022\": container with ID starting with 64d61561522a3da7aea642645fb52280d46f3551ad96c399cd8944825bc39022 not found: ID does not exist" Sep 10 11:25:19.459513 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.459487 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-j2zwc"] Sep 10 11:25:19.461769 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:19.461747 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-j2zwc"] Sep 10 11:25:20.171134 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:20.171101 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" path="/var/lib/kubelet/pods/7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05/volumes" Sep 10 11:25:20.171520 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:20.171489 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87dae763-33ee-4873-bb40-1ce5aec0779d" path="/var/lib/kubelet/pods/87dae763-33ee-4873-bb40-1ce5aec0779d/volumes" Sep 10 11:25:30.107809 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:30.107774 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:25:56.883474 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:56.883420 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:25:56.941218 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:56.941184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:25:57.241510 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:57.241473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:25:57.342557 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:25:57.342523 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:27:16.527976 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:27:16.527929 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:27:34.250883 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:27:34.250858 2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]" Sep 10 11:27:42.112184 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:27:42.112149 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:27:42.113623 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:27:42.113597 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:27:54.627148 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:27:54.627108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:27:54.728789 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:27:54.728751 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:27:54.827918 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:27:54.827888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:00.280456 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:00.280418 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:05.709292 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:05.709254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:05.796114 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:05.796076 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:05.939997 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:05.939955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:06.043291 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:06.043200 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:30.475466 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:30.475433 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:31.594958 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:31.594924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:31.679536 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:31.679477 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:31.967832 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:31.967797 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:32.128541 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:32.128507 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:28:50.483310 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:28:50.483259 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:12.259115 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:12.259078 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:15.024724 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:15.024684 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:15.414750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:15.414656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:15.639460 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:15.639427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:15.762629 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:15.762597 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:17.317716 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:17.317684 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:20.636624 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:20.636588 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:20.735647 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:20.735610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:29.105755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:29.105721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:29.202208 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:29.202174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:29.698665 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:29.698601 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:42.179810 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:42.179779 2560 scope.go:117] "RemoveContainer" containerID="224a1de2439687c83d3bdf3fa916cc69e5bacb2a12ef7a462ddb6574c6d4c244" Sep 10 11:30:42.187523 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:42.187498 2560 scope.go:117] "RemoveContainer" containerID="c65ccc119d9fb7bad2fad5f6a8d944576f1cafaee5a41ce8a61b5dbb33dd468d" Sep 10 11:30:43.955580 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:43.955542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:44.056222 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:44.056193 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:54.946879 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:54.946846 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:55.047052 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:55.047023 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:55.422409 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:55.422377 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:30:59.660616 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:30:59.660585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:05.412876 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:05.412834 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:05.513888 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:05.513851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:26.486668 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:26.486629 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:26.586411 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:26.586380 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:26.993918 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:26.993883 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:27.052949 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:27.052912 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:28.550512 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.550456 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:28.649501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.649460 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:28.683168 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683131 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g"] Sep 10 11:31:28.683485 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683472 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87dae763-33ee-4873-bb40-1ce5aec0779d" containerName="redis" Sep 10 11:31:28.683537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683486 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="87dae763-33ee-4873-bb40-1ce5aec0779d" containerName="redis" Sep 10 11:31:28.683537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683507 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" containerName="feast-0" Sep 10 11:31:28.683537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683513 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" containerName="feast-0" Sep 10 11:31:28.683537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683521 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" containerName="feast-1" Sep 10 11:31:28.683537 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683527 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" containerName="feast-1" Sep 10 11:31:28.683690 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683593 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="87dae763-33ee-4873-bb40-1ce5aec0779d" containerName="redis" Sep 10 11:31:28.683690 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.683604 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fdbabeb-9a06-4b51-acb7-a7baaaeb5a05" containerName="feast-1" Sep 10 11:31:28.687028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.687001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.689501 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.689478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 10 11:31:28.689659 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.689528 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 11:31:28.689659 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.689540 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 11:31:28.689816 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.689801 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 10 11:31:28.689883 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.689820 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 10 11:31:28.690593 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.690579 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 10 11:31:28.690658 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.690617 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-875b6\"" Sep 10 11:31:28.701401 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.701370 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g"] Sep 10 11:31:28.726442 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.726405 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:28.764222 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.764188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.764420 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.764237 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.764420 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.764330 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/78ef649f-e37a-4717-bc23-f7cd43c18d6e-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.764420 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.764374 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq2wz\" (UniqueName: \"kubernetes.io/projected/78ef649f-e37a-4717-bc23-f7cd43c18d6e-kube-api-access-qq2wz\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.764420 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.764408 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.764561 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.764425 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.797414 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.797375 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:28.865991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.865880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.865991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.865951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/78ef649f-e37a-4717-bc23-f7cd43c18d6e-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.866242 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.865991 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq2wz\" (UniqueName: \"kubernetes.io/projected/78ef649f-e37a-4717-bc23-f7cd43c18d6e-kube-api-access-qq2wz\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.866242 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.866129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.866242 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.866164 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.866242 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.866228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.866528 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.866486 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/78ef649f-e37a-4717-bc23-f7cd43c18d6e-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.868651 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.868622 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.868766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.868688 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.868766 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.868730 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.868841 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.868780 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.880726 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.880696 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq2wz\" (UniqueName: \"kubernetes.io/projected/78ef649f-e37a-4717-bc23-f7cd43c18d6e-kube-api-access-qq2wz\") pod \"feast-simple-feast-setup-57576b8fcf-4b82g\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:28.917915 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:28.917852 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:29.011874 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:29.011834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:29.017885 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:29.017853 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:31:29.294880 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:29.294854 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g"] Sep 10 11:31:29.297121 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:31:29.297092 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78ef649f_e37a_4717_bc23_f7cd43c18d6e.slice/crio-3720e4189b564e5a50fafef396fa97f4f9bbd1a943e48d0ee5d5b8302c48ade9 WatchSource:0}: Error finding container 3720e4189b564e5a50fafef396fa97f4f9bbd1a943e48d0ee5d5b8302c48ade9: Status 404 returned error can't find the container with id 3720e4189b564e5a50fafef396fa97f4f9bbd1a943e48d0ee5d5b8302c48ade9 Sep 10 11:31:29.299363 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:29.299342 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 11:31:29.955641 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:29.955609 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerStarted","Data":"3720e4189b564e5a50fafef396fa97f4f9bbd1a943e48d0ee5d5b8302c48ade9"} Sep 10 11:31:44.033576 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:44.033535 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerStarted","Data":"4db1db03bb039cfb8f256f47aa42a56e2a72354fe970d603530f5759f5b55be2"} Sep 10 11:31:49.055185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:49.055095 2560 generic.go:356] "Generic (PLEG): container finished" podID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerID="4db1db03bb039cfb8f256f47aa42a56e2a72354fe970d603530f5759f5b55be2" exitCode=0 Sep 10 11:31:49.055185 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:49.055166 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerDied","Data":"4db1db03bb039cfb8f256f47aa42a56e2a72354fe970d603530f5759f5b55be2"} Sep 10 11:31:50.062001 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:50.061961 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerStarted","Data":"7517eeba72078fc57c15d8cfbfa552aac5e21c5681f0cf485219c4f88ef75771"} Sep 10 11:31:56.092067 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:56.092035 2560 generic.go:356] "Generic (PLEG): container finished" podID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerID="7517eeba72078fc57c15d8cfbfa552aac5e21c5681f0cf485219c4f88ef75771" exitCode=0 Sep 10 11:31:56.092472 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:56.092111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerDied","Data":"7517eeba72078fc57c15d8cfbfa552aac5e21c5681f0cf485219c4f88ef75771"} Sep 10 11:31:57.103153 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:57.103109 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerStarted","Data":"0c4af6ad14e94add5376d7a24217c1919be1315a6f3d7c303ddda341b608f4d9"} Sep 10 11:31:57.103670 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:57.103177 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerStarted","Data":"af9acfab5cd3812678af88178a6eece64ad309354d102ab27b0d7829d31abbbd"} Sep 10 11:31:57.103670 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:57.103201 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerStarted","Data":"0387ffc764ad0cd5dcc629ad2bfa16ff85321c419be289e317a57e904bfad7c6"} Sep 10 11:31:57.103670 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:57.103221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerStarted","Data":"7a833f9145a7a5c413dff95d64085f7379dbdd606c51081ead4873392e0a9f22"} Sep 10 11:31:57.130321 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:57.130239 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podStartSLOduration=14.805357931 podStartE2EDuration="29.130216574s" totalImagesPullingTime="14.324858643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:31:57.128815286 +0000 UTC m=+1155.487871513" watchObservedRunningTime="2026-09-10 11:31:57.130216574 +0000 UTC m=+1155.489272782" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013028 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013096 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013117 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013137 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013154 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013173 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013191 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.013273 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.013208 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:31:59.015594 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.015410 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 10 11:31:59.015594 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.015442 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 10 11:31:59.015594 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.015491 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused" Sep 10 11:31:59.015594 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:31:59.015447 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 11:32:02.012993 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:02.012946 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused" Sep 10 11:32:02.013622 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:02.012947 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused" Sep 10 11:32:02.013746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:02.012949 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 10 11:32:02.013746 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:02.012988 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 11:32:05.013982 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.013899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:05.014359 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.014016 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:05.014359 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.014239 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:05.014941 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.014916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:05.015055 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.014987 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:05.015092 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.015080 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:05.019637 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.019618 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:05.159368 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:05.159330 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:32:42.142587 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:42.142556 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:32:42.148971 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:32:42.148937 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:33:07.421500 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.421463 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g"] Sep 10 11:33:07.423579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.421713 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="registry" containerID="cri-o://7a833f9145a7a5c413dff95d64085f7379dbdd606c51081ead4873392e0a9f22" gracePeriod=30 Sep 10 11:33:07.423579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.421748 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" containerID="cri-o://0c4af6ad14e94add5376d7a24217c1919be1315a6f3d7c303ddda341b608f4d9" gracePeriod=30 Sep 10 11:33:07.423579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.421763 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="offline" containerID="cri-o://af9acfab5cd3812678af88178a6eece64ad309354d102ab27b0d7829d31abbbd" gracePeriod=30 Sep 10 11:33:07.423579 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.421789 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" containerID="cri-o://0387ffc764ad0cd5dcc629ad2bfa16ff85321c419be289e317a57e904bfad7c6" gracePeriod=30 Sep 10 11:33:07.449595 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.449538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:33:07.845266 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.845174 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx"] Sep 10 11:33:07.848559 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.848535 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:07.851005 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.850969 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-75tzt\"" Sep 10 11:33:07.860603 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.860574 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx"] Sep 10 11:33:07.940767 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.940721 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:07.940968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.940782 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:07.940968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.940850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef64c0d-633b-4691-afa7-13fe8abac41d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:07.940968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.940923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqht2\" (UniqueName: \"kubernetes.io/projected/2ef64c0d-633b-4691-afa7-13fe8abac41d-kube-api-access-qqht2\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:07.940968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.940942 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:07.941175 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:07.941011 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.042361 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.042290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqht2\" (UniqueName: \"kubernetes.io/projected/2ef64c0d-633b-4691-afa7-13fe8abac41d-kube-api-access-qqht2\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.042361 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.042354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.042378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.042444 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.042494 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.042501 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.042530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef64c0d-633b-4691-afa7-13fe8abac41d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.042544 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.042564 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls podName:2ef64c0d-633b-4691-afa7-13fe8abac41d nodeName:}" failed. No retries permitted until 2026-09-10 11:33:08.542540565 +0000 UTC m=+1226.901596774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-x56tx" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d") : secret "feast-simple-feast-setup-registry-tls" not found Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.042636 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls podName:2ef64c0d-633b-4691-afa7-13fe8abac41d nodeName:}" failed. No retries permitted until 2026-09-10 11:33:08.542589049 +0000 UTC m=+1226.901645249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-x56tx" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d") : secret "feast-simple-feast-setup-ui-tls" not found Sep 10 11:33:08.042642 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.042638 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 10 11:33:08.043110 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.042677 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls podName:2ef64c0d-633b-4691-afa7-13fe8abac41d nodeName:}" failed. No retries permitted until 2026-09-10 11:33:08.542666613 +0000 UTC m=+1226.901722816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-x56tx" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d") : secret "feast-simple-feast-setup-offline-tls" not found Sep 10 11:33:08.043110 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.043023 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef64c0d-633b-4691-afa7-13fe8abac41d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.044964 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.044937 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.054135 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.054100 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqht2\" (UniqueName: \"kubernetes.io/projected/2ef64c0d-633b-4691-afa7-13fe8abac41d-kube-api-access-qqht2\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.429314 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.429267 2560 generic.go:356] "Generic (PLEG): container finished" podID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerID="0c4af6ad14e94add5376d7a24217c1919be1315a6f3d7c303ddda341b608f4d9" exitCode=0 Sep 10 11:33:08.429314 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.429293 2560 generic.go:356] "Generic (PLEG): container finished" podID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerID="0387ffc764ad0cd5dcc629ad2bfa16ff85321c419be289e317a57e904bfad7c6" exitCode=0 Sep 10 11:33:08.429751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.429332 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerDied","Data":"0c4af6ad14e94add5376d7a24217c1919be1315a6f3d7c303ddda341b608f4d9"} Sep 10 11:33:08.429751 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.429384 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerDied","Data":"0387ffc764ad0cd5dcc629ad2bfa16ff85321c419be289e317a57e904bfad7c6"} Sep 10 11:33:08.547521 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.547476 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.547732 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.547551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.547732 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.547584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.547732 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.547711 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 10 11:33:08.547900 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:33:08.547790 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls podName:2ef64c0d-633b-4691-afa7-13fe8abac41d nodeName:}" failed. No retries permitted until 2026-09-10 11:33:09.547771827 +0000 UTC m=+1227.906828015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-x56tx" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d") : secret "feast-simple-feast-setup-ui-tls" not found Sep 10 11:33:08.550039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.550003 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.550039 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.550028 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:08.761524 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:08.761426 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:33:09.559570 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:09.559530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:09.562063 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:09.562039 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-x56tx\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:09.658750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:09.658704 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:09.943911 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:09.943882 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx"] Sep 10 11:33:09.945682 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:33:09.945654 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ef64c0d_633b_4691_afa7_13fe8abac41d.slice/crio-5f08121b3daffd6a499baf502c03b97df05f5bc8d8c7c8a1ab5e4f1b039d73db WatchSource:0}: Error finding container 5f08121b3daffd6a499baf502c03b97df05f5bc8d8c7c8a1ab5e4f1b039d73db: Status 404 returned error can't find the container with id 5f08121b3daffd6a499baf502c03b97df05f5bc8d8c7c8a1ab5e4f1b039d73db Sep 10 11:33:10.439169 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:10.439134 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerStarted","Data":"11d33ad769c9475acf0b9b5da24b979d984e9b2ca6595bcefef7e531c28cfa2f"} Sep 10 11:33:10.439169 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:10.439179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerStarted","Data":"5f08121b3daffd6a499baf502c03b97df05f5bc8d8c7c8a1ab5e4f1b039d73db"} Sep 10 11:33:15.015741 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:15.015690 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 10 11:33:15.156731 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:15.156684 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 11:33:15.464316 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:15.464263 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerID="11d33ad769c9475acf0b9b5da24b979d984e9b2ca6595bcefef7e531c28cfa2f" exitCode=0 Sep 10 11:33:15.464496 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:15.464336 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerDied","Data":"11d33ad769c9475acf0b9b5da24b979d984e9b2ca6595bcefef7e531c28cfa2f"} Sep 10 11:33:16.471274 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:16.471237 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerStarted","Data":"e35e04dce4a33b33cd4b0f83c40259bd1d768d924b8336793e3ab6d2f3b38f29"} Sep 10 11:33:22.505740 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:22.505702 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerID="e35e04dce4a33b33cd4b0f83c40259bd1d768d924b8336793e3ab6d2f3b38f29" exitCode=0 Sep 10 11:33:22.506146 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:22.505777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerDied","Data":"e35e04dce4a33b33cd4b0f83c40259bd1d768d924b8336793e3ab6d2f3b38f29"} Sep 10 11:33:23.516941 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:23.516896 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerStarted","Data":"447a60ab574df3fbca8d6f078dff1c2afa6bd939c5b66da89e9dd7ae28e609fb"} Sep 10 11:33:23.517489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:23.516965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerStarted","Data":"6e603e67f812a27c547ab290a68255e8c74606f8342336813b1df65ab409499f"} Sep 10 11:33:23.517489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:23.516989 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerStarted","Data":"29354f5388240e52de43a38fc28e3e7e2134954dd29763c6c11a53ceb9dedbd5"} Sep 10 11:33:23.517489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:23.517011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerStarted","Data":"11920f556bbf8f9cc82d38c697807c7784eb9cc06e52e8fca081088c37cd1f4e"} Sep 10 11:33:23.549260 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:23.549190 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podStartSLOduration=16.549164714 podStartE2EDuration="16.549164714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:33:23.548598669 +0000 UTC m=+1241.907654876" watchObservedRunningTime="2026-09-10 11:33:23.549164714 +0000 UTC m=+1241.908220981" Sep 10 11:33:24.659412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.659260 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:24.659412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.659339 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:24.659412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.659360 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:24.659412 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.659382 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:24.661446 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.661401 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused" Sep 10 11:33:24.661446 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.661427 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 10 11:33:24.661685 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.661488 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused" Sep 10 11:33:24.661685 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:24.661437 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 10 11:33:25.014870 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:25.014819 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 10 11:33:25.156547 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:25.156500 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 11:33:27.660221 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:27.660070 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused" Sep 10 11:33:27.660221 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:27.660092 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 10 11:33:27.660221 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:27.660159 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 10 11:33:27.660221 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:27.660091 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused" Sep 10 11:33:29.659795 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:29.659742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:29.660284 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:29.659818 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:29.660284 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:29.659840 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:29.660284 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:29.659859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:30.659554 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:30.659508 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Sep 10 11:33:30.659752 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:30.659508 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused" Sep 10 11:33:30.659752 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:30.659599 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused" Sep 10 11:33:30.659752 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:30.659526 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 10 11:33:33.660103 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:33.660072 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:33.660472 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:33.660216 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:33.660578 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:33.660558 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:33.660901 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:33.660877 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:33.661028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:33.661006 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:33.661125 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:33.661110 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:33.665366 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:33.665347 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:34.586061 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:34.585977 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:33:35.015197 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:35.015151 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused" Sep 10 11:33:35.015643 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:35.015346 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:33:35.157046 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:35.157009 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused" Sep 10 11:33:35.157228 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:35.157138 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:33:37.598317 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:37.598277 2560 generic.go:356] "Generic (PLEG): container finished" podID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerID="af9acfab5cd3812678af88178a6eece64ad309354d102ab27b0d7829d31abbbd" exitCode=137 Sep 10 11:33:37.598317 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:37.598312 2560 generic.go:356] "Generic (PLEG): container finished" podID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerID="7a833f9145a7a5c413dff95d64085f7379dbdd606c51081ead4873392e0a9f22" exitCode=137 Sep 10 11:33:37.598705 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:37.598343 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerDied","Data":"af9acfab5cd3812678af88178a6eece64ad309354d102ab27b0d7829d31abbbd"} Sep 10 11:33:37.598705 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:37.598389 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerDied","Data":"7a833f9145a7a5c413dff95d64085f7379dbdd606c51081ead4873392e0a9f22"} Sep 10 11:33:38.081472 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.081435 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:33:38.144070 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.144032 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-offline-tls\") pod \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " Sep 10 11:33:38.144070 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.144077 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-online-tls\") pod \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " Sep 10 11:33:38.144362 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.144099 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-ui-tls\") pod \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " Sep 10 11:33:38.144362 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.144133 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq2wz\" (UniqueName: \"kubernetes.io/projected/78ef649f-e37a-4717-bc23-f7cd43c18d6e-kube-api-access-qq2wz\") pod \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " Sep 10 11:33:38.144362 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.144173 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-registry-tls\") pod \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " Sep 10 11:33:38.144362 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.144195 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/78ef649f-e37a-4717-bc23-f7cd43c18d6e-feast-data\") pod \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\" (UID: \"78ef649f-e37a-4717-bc23-f7cd43c18d6e\") " Sep 10 11:33:38.144907 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.144858 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ef649f-e37a-4717-bc23-f7cd43c18d6e-feast-data" (OuterVolumeSpecName: "feast-data") pod "78ef649f-e37a-4717-bc23-f7cd43c18d6e" (UID: "78ef649f-e37a-4717-bc23-f7cd43c18d6e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 11:33:38.146616 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.146589 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ef649f-e37a-4717-bc23-f7cd43c18d6e-kube-api-access-qq2wz" (OuterVolumeSpecName: "kube-api-access-qq2wz") pod "78ef649f-e37a-4717-bc23-f7cd43c18d6e" (UID: "78ef649f-e37a-4717-bc23-f7cd43c18d6e"). InnerVolumeSpecName "kube-api-access-qq2wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:33:38.146821 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.146788 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-online-tls" (OuterVolumeSpecName: "online-tls") pod "78ef649f-e37a-4717-bc23-f7cd43c18d6e" (UID: "78ef649f-e37a-4717-bc23-f7cd43c18d6e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:33:38.146924 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.146861 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "78ef649f-e37a-4717-bc23-f7cd43c18d6e" (UID: "78ef649f-e37a-4717-bc23-f7cd43c18d6e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:33:38.146924 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.146896 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "78ef649f-e37a-4717-bc23-f7cd43c18d6e" (UID: "78ef649f-e37a-4717-bc23-f7cd43c18d6e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:33:38.146924 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.146910 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "78ef649f-e37a-4717-bc23-f7cd43c18d6e" (UID: "78ef649f-e37a-4717-bc23-f7cd43c18d6e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:33:38.245001 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.244899 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-offline-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:33:38.245001 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.244962 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-online-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:33:38.245001 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.244971 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-ui-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:33:38.245001 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.244981 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq2wz\" (UniqueName: \"kubernetes.io/projected/78ef649f-e37a-4717-bc23-f7cd43c18d6e-kube-api-access-qq2wz\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:33:38.245001 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.244993 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/78ef649f-e37a-4717-bc23-f7cd43c18d6e-registry-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:33:38.245001 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.245002 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/78ef649f-e37a-4717-bc23-f7cd43c18d6e-feast-data\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:33:38.606460 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.606358 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" event={"ID":"78ef649f-e37a-4717-bc23-f7cd43c18d6e","Type":"ContainerDied","Data":"3720e4189b564e5a50fafef396fa97f4f9bbd1a943e48d0ee5d5b8302c48ade9"} Sep 10 11:33:38.606460 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.606391 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g" Sep 10 11:33:38.606460 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.606433 2560 scope.go:117] "RemoveContainer" containerID="0c4af6ad14e94add5376d7a24217c1919be1315a6f3d7c303ddda341b608f4d9" Sep 10 11:33:38.615637 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.615612 2560 scope.go:117] "RemoveContainer" containerID="af9acfab5cd3812678af88178a6eece64ad309354d102ab27b0d7829d31abbbd" Sep 10 11:33:38.624254 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.624231 2560 scope.go:117] "RemoveContainer" containerID="0387ffc764ad0cd5dcc629ad2bfa16ff85321c419be289e317a57e904bfad7c6" Sep 10 11:33:38.630711 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.630684 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g"] Sep 10 11:33:38.633239 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.633220 2560 scope.go:117] "RemoveContainer" containerID="7a833f9145a7a5c413dff95d64085f7379dbdd606c51081ead4873392e0a9f22" Sep 10 11:33:38.637238 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.637212 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-4b82g"] Sep 10 11:33:38.642227 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.642203 2560 scope.go:117] "RemoveContainer" containerID="7517eeba72078fc57c15d8cfbfa552aac5e21c5681f0cf485219c4f88ef75771" Sep 10 11:33:38.650440 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:38.650419 2560 scope.go:117] "RemoveContainer" containerID="4db1db03bb039cfb8f256f47aa42a56e2a72354fe970d603530f5759f5b55be2" Sep 10 11:33:40.172053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:33:40.172018 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" path="/var/lib/kubelet/pods/78ef649f-e37a-4717-bc23-f7cd43c18d6e/volumes" Sep 10 11:34:30.677903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:30.677867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:36.087171 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.087121 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:36.185238 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.185186 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:36.560696 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.560660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:36.659498 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.659464 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:36.762827 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.762788 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz"] Sep 10 11:34:36.763201 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763182 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="registry" Sep 10 11:34:36.763288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763204 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="registry" Sep 10 11:34:36.763288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763224 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" Sep 10 11:34:36.763288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763232 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" Sep 10 11:34:36.763288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763257 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="feast-init" Sep 10 11:34:36.763288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763266 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="feast-init" Sep 10 11:34:36.763288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763277 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" Sep 10 11:34:36.763288 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763286 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763314 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="offline" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763323 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="offline" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763343 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="feast-apply" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763353 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="feast-apply" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763439 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="registry" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763452 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="offline" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763459 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="ui" Sep 10 11:34:36.763678 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.763473 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ef649f-e37a-4717-bc23-f7cd43c18d6e" containerName="online" Sep 10 11:34:36.767136 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.767114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.769622 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.769596 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 10 11:34:36.769742 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.769671 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 10 11:34:36.769742 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.769658 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 10 11:34:36.769851 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.769737 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 10 11:34:36.769962 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.769946 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-xf747\"" Sep 10 11:34:36.770018 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.769947 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 10 11:34:36.770064 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.770039 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 10 11:34:36.778549 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.778524 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz"] Sep 10 11:34:36.836261 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.836171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:36.857662 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.857631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.857888 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.857674 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce14a720-757c-4da7-8c26-aca6fe212ce2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.857888 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.857769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce14a720-757c-4da7-8c26-aca6fe212ce2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.857888 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.857810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.858073 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.857941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkv88\" (UniqueName: \"kubernetes.io/projected/ce14a720-757c-4da7-8c26-aca6fe212ce2-kube-api-access-wkv88\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.858073 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.858005 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.915679 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.915640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:36.959087 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.959033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.959346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.959136 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkv88\" (UniqueName: \"kubernetes.io/projected/ce14a720-757c-4da7-8c26-aca6fe212ce2-kube-api-access-wkv88\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.959346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.959177 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.959346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.959200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.959346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.959222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce14a720-757c-4da7-8c26-aca6fe212ce2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.959346 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.959256 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce14a720-757c-4da7-8c26-aca6fe212ce2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.959635 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:34:36.959359 2560 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 10 11:34:36.959635 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:34:36.959480 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls podName:ce14a720-757c-4da7-8c26-aca6fe212ce2 nodeName:}" failed. No retries permitted until 2026-09-10 11:34:37.459453949 +0000 UTC m=+1315.818510152 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" (UID: "ce14a720-757c-4da7-8c26-aca6fe212ce2") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 10 11:34:36.959747 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.959702 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce14a720-757c-4da7-8c26-aca6fe212ce2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.960045 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.960022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce14a720-757c-4da7-8c26-aca6fe212ce2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.961684 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.961664 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.961814 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.961793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:36.967488 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:36.967462 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkv88\" (UniqueName: \"kubernetes.io/projected/ce14a720-757c-4da7-8c26-aca6fe212ce2-kube-api-access-wkv88\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:37.073430 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:37.073394 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:34:37.463894 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:37.463853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:37.466418 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:37.466390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8n7jz\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:37.677096 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:37.677056 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:34:37.958566 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:37.958538 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz"] Sep 10 11:34:37.960312 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:34:37.960273 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce14a720_757c_4da7_8c26_aca6fe212ce2.slice/crio-4380e27c2d3944a6a0928d1b84ef3167baa5fd352e5d3511399065d1c1598b92 WatchSource:0}: Error finding container 4380e27c2d3944a6a0928d1b84ef3167baa5fd352e5d3511399065d1c1598b92: Status 404 returned error can't find the container with id 4380e27c2d3944a6a0928d1b84ef3167baa5fd352e5d3511399065d1c1598b92 Sep 10 11:34:38.873032 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:38.872996 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerStarted","Data":"ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74"} Sep 10 11:34:38.873437 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:38.873044 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerStarted","Data":"4380e27c2d3944a6a0928d1b84ef3167baa5fd352e5d3511399065d1c1598b92"} Sep 10 11:34:43.896642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:43.896606 2560 generic.go:356] "Generic (PLEG): container finished" podID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerID="ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74" exitCode=0 Sep 10 11:34:43.897041 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:43.896664 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerDied","Data":"ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74"} Sep 10 11:34:44.903022 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:34:44.902986 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerStarted","Data":"cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac"} Sep 10 11:35:06.999044 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:06.999004 2560 generic.go:356] "Generic (PLEG): container finished" podID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerID="cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac" exitCode=0 Sep 10 11:35:06.999505 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:06.999080 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerDied","Data":"cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac"} Sep 10 11:35:08.007085 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:08.007037 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerStarted","Data":"e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3"} Sep 10 11:35:08.007591 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:08.007103 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerStarted","Data":"942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94"} Sep 10 11:35:08.007591 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:08.007122 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerStarted","Data":"4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66"} Sep 10 11:35:08.033616 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:08.033546 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podStartSLOduration=32.033522058 podStartE2EDuration="32.033522058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:34:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:35:08.031209233 +0000 UTC m=+1346.390265439" watchObservedRunningTime="2026-09-10 11:35:08.033522058 +0000 UTC m=+1346.392578267" Sep 10 11:35:10.677839 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:10.677794 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:10.678441 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:10.678415 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:10.678441 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:10.678449 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:10.679794 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:10.679756 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 10 11:35:10.679895 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:10.679803 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 10 11:35:10.679997 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:10.679970 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused" Sep 10 11:35:13.677582 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:13.677532 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 10 11:35:13.678008 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:13.677542 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused" Sep 10 11:35:13.678008 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:13.677547 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 10 11:35:16.678794 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:16.678756 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:16.679261 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:16.679223 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:16.679261 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:16.679260 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:16.679452 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:16.679432 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:17.057522 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:17.057418 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:17.058028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:17.058005 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:17.678002 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:17.677965 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:17.681495 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:17.681449 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 11:35:18.165349 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:18.165252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:35:20.379103 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:20.379070 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:35:21.126148 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:35:21.126112 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:36:18.682458 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:18.682412 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 11:36:22.738693 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.738650 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:36:22.748903 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.748869 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz"] Sep 10 11:36:22.749176 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.749135 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" containerID="cri-o://4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66" gracePeriod=30 Sep 10 11:36:22.749267 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.749167 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" containerID="cri-o://e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3" gracePeriod=30 Sep 10 11:36:22.749267 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.749192 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="offline" containerID="cri-o://942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94" gracePeriod=30 Sep 10 11:36:22.803053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.803013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:36:22.942888 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.942845 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:36:22.948348 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.948315 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx"] Sep 10 11:36:22.948750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.948597 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="registry" containerID="cri-o://11920f556bbf8f9cc82d38c697807c7784eb9cc06e52e8fca081088c37cd1f4e" gracePeriod=30 Sep 10 11:36:22.948750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.948648 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="offline" containerID="cri-o://6e603e67f812a27c547ab290a68255e8c74606f8342336813b1df65ab409499f" gracePeriod=30 Sep 10 11:36:22.948750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.948672 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" containerID="cri-o://29354f5388240e52de43a38fc28e3e7e2134954dd29763c6c11a53ceb9dedbd5" gracePeriod=30 Sep 10 11:36:22.948980 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:22.948647 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" containerID="cri-o://447a60ab574df3fbca8d6f078dff1c2afa6bd939c5b66da89e9dd7ae28e609fb" gracePeriod=30 Sep 10 11:36:23.044066 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:23.043974 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:36:23.102088 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:23.102048 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:36:23.661530 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:23.661489 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused" Sep 10 11:36:24.359645 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.359613 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerID="447a60ab574df3fbca8d6f078dff1c2afa6bd939c5b66da89e9dd7ae28e609fb" exitCode=0 Sep 10 11:36:24.359645 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.359638 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerID="29354f5388240e52de43a38fc28e3e7e2134954dd29763c6c11a53ceb9dedbd5" exitCode=0 Sep 10 11:36:24.360190 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.359680 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerDied","Data":"447a60ab574df3fbca8d6f078dff1c2afa6bd939c5b66da89e9dd7ae28e609fb"} Sep 10 11:36:24.360190 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.359724 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerDied","Data":"29354f5388240e52de43a38fc28e3e7e2134954dd29763c6c11a53ceb9dedbd5"} Sep 10 11:36:24.362246 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.362225 2560 generic.go:356] "Generic (PLEG): container finished" podID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerID="e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3" exitCode=0 Sep 10 11:36:24.362246 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.362244 2560 generic.go:356] "Generic (PLEG): container finished" podID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerID="4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66" exitCode=0 Sep 10 11:36:24.362404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.362283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerDied","Data":"e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3"} Sep 10 11:36:24.362404 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.362325 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerDied","Data":"4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66"} Sep 10 11:36:24.583764 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:24.583722 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 10 11:36:26.679500 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:26.679450 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 10 11:36:28.108937 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:28.108904 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:36:30.380693 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:30.380644 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 10 11:36:33.661510 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:33.661443 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused" Sep 10 11:36:34.583551 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:34.583459 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 10 11:36:36.680258 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:36.680212 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 10 11:36:40.380440 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:40.380393 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 10 11:36:43.661697 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:43.661628 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused" Sep 10 11:36:43.662138 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:43.661814 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:36:44.583871 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:44.583818 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Sep 10 11:36:44.584066 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:44.583948 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:36:46.680127 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:46.680072 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused" Sep 10 11:36:46.680563 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:46.680218 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:36:50.380436 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:50.380394 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused" Sep 10 11:36:50.380833 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:50.380517 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:36:53.400219 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.400194 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:36:53.494833 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.494796 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerID="6e603e67f812a27c547ab290a68255e8c74606f8342336813b1df65ab409499f" exitCode=137 Sep 10 11:36:53.494833 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.494821 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerID="11920f556bbf8f9cc82d38c697807c7784eb9cc06e52e8fca081088c37cd1f4e" exitCode=137 Sep 10 11:36:53.495083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.494868 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerDied","Data":"6e603e67f812a27c547ab290a68255e8c74606f8342336813b1df65ab409499f"} Sep 10 11:36:53.495083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.494921 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerDied","Data":"11920f556bbf8f9cc82d38c697807c7784eb9cc06e52e8fca081088c37cd1f4e"} Sep 10 11:36:53.497396 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.497362 2560 generic.go:356] "Generic (PLEG): container finished" podID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerID="942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94" exitCode=137 Sep 10 11:36:53.497539 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.497415 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerDied","Data":"942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94"} Sep 10 11:36:53.497539 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.497460 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" event={"ID":"ce14a720-757c-4da7-8c26-aca6fe212ce2","Type":"ContainerDied","Data":"4380e27c2d3944a6a0928d1b84ef3167baa5fd352e5d3511399065d1c1598b92"} Sep 10 11:36:53.497539 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.497491 2560 scope.go:117] "RemoveContainer" containerID="e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3" Sep 10 11:36:53.497539 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.497420 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz" Sep 10 11:36:53.507948 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.507926 2560 scope.go:117] "RemoveContainer" containerID="942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94" Sep 10 11:36:53.517224 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.517199 2560 scope.go:117] "RemoveContainer" containerID="4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66" Sep 10 11:36:53.525882 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.525855 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce14a720-757c-4da7-8c26-aca6fe212ce2-registry-tls\") pod \"ce14a720-757c-4da7-8c26-aca6fe212ce2\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " Sep 10 11:36:53.526053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.525892 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce14a720-757c-4da7-8c26-aca6fe212ce2-feast-data\") pod \"ce14a720-757c-4da7-8c26-aca6fe212ce2\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " Sep 10 11:36:53.526053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.525928 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-offline-tls\") pod \"ce14a720-757c-4da7-8c26-aca6fe212ce2\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " Sep 10 11:36:53.526053 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.525953 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls\") pod \"ce14a720-757c-4da7-8c26-aca6fe212ce2\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " Sep 10 11:36:53.526755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.526292 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce14a720-757c-4da7-8c26-aca6fe212ce2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ce14a720-757c-4da7-8c26-aca6fe212ce2" (UID: "ce14a720-757c-4da7-8c26-aca6fe212ce2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 11:36:53.526755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.526349 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-online-tls\") pod \"ce14a720-757c-4da7-8c26-aca6fe212ce2\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " Sep 10 11:36:53.526755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.526397 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkv88\" (UniqueName: \"kubernetes.io/projected/ce14a720-757c-4da7-8c26-aca6fe212ce2-kube-api-access-wkv88\") pod \"ce14a720-757c-4da7-8c26-aca6fe212ce2\" (UID: \"ce14a720-757c-4da7-8c26-aca6fe212ce2\") " Sep 10 11:36:53.526755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.526444 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce14a720-757c-4da7-8c26-aca6fe212ce2-feast-data" (OuterVolumeSpecName: "feast-data") pod "ce14a720-757c-4da7-8c26-aca6fe212ce2" (UID: "ce14a720-757c-4da7-8c26-aca6fe212ce2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 11:36:53.526755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.526716 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ce14a720-757c-4da7-8c26-aca6fe212ce2-registry-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.526755 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.526732 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ce14a720-757c-4da7-8c26-aca6fe212ce2-feast-data\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.527216 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.526963 2560 scope.go:117] "RemoveContainer" containerID="cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac" Sep 10 11:36:53.529295 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.529264 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ce14a720-757c-4da7-8c26-aca6fe212ce2" (UID: "ce14a720-757c-4da7-8c26-aca6fe212ce2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:36:53.529295 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.529280 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ce14a720-757c-4da7-8c26-aca6fe212ce2" (UID: "ce14a720-757c-4da7-8c26-aca6fe212ce2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:36:53.529489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.529374 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce14a720-757c-4da7-8c26-aca6fe212ce2-kube-api-access-wkv88" (OuterVolumeSpecName: "kube-api-access-wkv88") pod "ce14a720-757c-4da7-8c26-aca6fe212ce2" (UID: "ce14a720-757c-4da7-8c26-aca6fe212ce2"). InnerVolumeSpecName "kube-api-access-wkv88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:36:53.529489 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.529455 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-online-tls" (OuterVolumeSpecName: "online-tls") pod "ce14a720-757c-4da7-8c26-aca6fe212ce2" (UID: "ce14a720-757c-4da7-8c26-aca6fe212ce2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:36:53.537042 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.537004 2560 scope.go:117] "RemoveContainer" containerID="ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74" Sep 10 11:36:53.547487 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.547460 2560 scope.go:117] "RemoveContainer" containerID="e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3" Sep 10 11:36:53.547820 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:36:53.547796 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3\": container with ID starting with e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3 not found: ID does not exist" containerID="e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3" Sep 10 11:36:53.547872 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.547832 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3"} err="failed to get container status \"e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3\": rpc error: code = NotFound desc = could not find container \"e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3\": container with ID starting with e1f6c586517a515efb3ea10c039d3bcc16b08244bb4347010b8ff4dbd4c56ea3 not found: ID does not exist" Sep 10 11:36:53.547872 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.547852 2560 scope.go:117] "RemoveContainer" containerID="942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94" Sep 10 11:36:53.548141 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:36:53.548114 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94\": container with ID starting with 942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94 not found: ID does not exist" containerID="942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94" Sep 10 11:36:53.548188 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.548153 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94"} err="failed to get container status \"942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94\": rpc error: code = NotFound desc = could not find container \"942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94\": container with ID starting with 942b0f8c8f5604e76693f04a0d6aa79ef5d4f2f8f764040a4a433d066f620c94 not found: ID does not exist" Sep 10 11:36:53.548188 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.548177 2560 scope.go:117] "RemoveContainer" containerID="4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66" Sep 10 11:36:53.548432 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:36:53.548415 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66\": container with ID starting with 4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66 not found: ID does not exist" containerID="4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66" Sep 10 11:36:53.548485 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.548436 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66"} err="failed to get container status \"4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66\": rpc error: code = NotFound desc = could not find container \"4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66\": container with ID starting with 4d015e93ef018d8ebf2f643a9515ead09749860f6365d97d9f1715a290e77c66 not found: ID does not exist" Sep 10 11:36:53.548485 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.548449 2560 scope.go:117] "RemoveContainer" containerID="cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac" Sep 10 11:36:53.548668 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:36:53.548649 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac\": container with ID starting with cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac not found: ID does not exist" containerID="cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac" Sep 10 11:36:53.548723 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.548672 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac"} err="failed to get container status \"cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac\": rpc error: code = NotFound desc = could not find container \"cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac\": container with ID starting with cd1489547a603a7e90477f5dafd8e487e6564fa2aaa7019611561a5d23eae1ac not found: ID does not exist" Sep 10 11:36:53.548723 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.548684 2560 scope.go:117] "RemoveContainer" containerID="ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74" Sep 10 11:36:53.548946 ip-10-0-131-79 kubenswrapper[2560]: E0910 11:36:53.548930 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74\": container with ID starting with ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74 not found: ID does not exist" containerID="ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74" Sep 10 11:36:53.548991 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.548949 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74"} err="failed to get container status \"ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74\": rpc error: code = NotFound desc = could not find container \"ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74\": container with ID starting with ffcc8fcc8ec4798d64ec5373d63350e81527dbc66e6235bc898a2a85cd70cf74 not found: ID does not exist" Sep 10 11:36:53.594163 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.594136 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:36:53.627683 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.627654 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-offline-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.627683 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.627679 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-ui-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.627683 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.627689 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ce14a720-757c-4da7-8c26-aca6fe212ce2-online-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.627898 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.627698 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkv88\" (UniqueName: \"kubernetes.io/projected/ce14a720-757c-4da7-8c26-aca6fe212ce2-kube-api-access-wkv88\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.728958 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.728903 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls\") pod \"2ef64c0d-633b-4691-afa7-13fe8abac41d\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " Sep 10 11:36:53.728958 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.728973 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-online-tls\") pod \"2ef64c0d-633b-4691-afa7-13fe8abac41d\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " Sep 10 11:36:53.729204 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.728992 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef64c0d-633b-4691-afa7-13fe8abac41d-feast-data\") pod \"2ef64c0d-633b-4691-afa7-13fe8abac41d\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " Sep 10 11:36:53.729204 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.729016 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls\") pod \"2ef64c0d-633b-4691-afa7-13fe8abac41d\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " Sep 10 11:36:53.729204 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.729046 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqht2\" (UniqueName: \"kubernetes.io/projected/2ef64c0d-633b-4691-afa7-13fe8abac41d-kube-api-access-qqht2\") pod \"2ef64c0d-633b-4691-afa7-13fe8abac41d\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " Sep 10 11:36:53.729204 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.729072 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls\") pod \"2ef64c0d-633b-4691-afa7-13fe8abac41d\" (UID: \"2ef64c0d-633b-4691-afa7-13fe8abac41d\") " Sep 10 11:36:53.729682 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.729645 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ef64c0d-633b-4691-afa7-13fe8abac41d-feast-data" (OuterVolumeSpecName: "feast-data") pod "2ef64c0d-633b-4691-afa7-13fe8abac41d" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 11:36:53.731194 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.731162 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-online-tls" (OuterVolumeSpecName: "online-tls") pod "2ef64c0d-633b-4691-afa7-13fe8abac41d" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:36:53.731330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.731195 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2ef64c0d-633b-4691-afa7-13fe8abac41d" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:36:53.731330 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.731230 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2ef64c0d-633b-4691-afa7-13fe8abac41d" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:36:53.731559 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.731542 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2ef64c0d-633b-4691-afa7-13fe8abac41d" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 11:36:53.731642 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.731589 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef64c0d-633b-4691-afa7-13fe8abac41d-kube-api-access-qqht2" (OuterVolumeSpecName: "kube-api-access-qqht2") pod "2ef64c0d-633b-4691-afa7-13fe8abac41d" (UID: "2ef64c0d-633b-4691-afa7-13fe8abac41d"). InnerVolumeSpecName "kube-api-access-qqht2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 11:36:53.820693 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.820656 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz"] Sep 10 11:36:53.826804 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.826779 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8n7jz"] Sep 10 11:36:53.830811 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.830786 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqht2\" (UniqueName: \"kubernetes.io/projected/2ef64c0d-633b-4691-afa7-13fe8abac41d-kube-api-access-qqht2\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.830811 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.830808 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-offline-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.830961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.830819 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-registry-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.830961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.830826 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-online-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.830961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.830834 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ef64c0d-633b-4691-afa7-13fe8abac41d-feast-data\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:53.830961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:53.830842 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ef64c0d-633b-4691-afa7-13fe8abac41d-ui-tls\") on node \"ip-10-0-131-79.ec2.internal\" DevicePath \"\"" Sep 10 11:36:54.171780 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.171750 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" path="/var/lib/kubelet/pods/ce14a720-757c-4da7-8c26-aca6fe212ce2/volumes" Sep 10 11:36:54.504785 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.504694 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" event={"ID":"2ef64c0d-633b-4691-afa7-13fe8abac41d","Type":"ContainerDied","Data":"5f08121b3daffd6a499baf502c03b97df05f5bc8d8c7c8a1ab5e4f1b039d73db"} Sep 10 11:36:54.504785 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.504710 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx" Sep 10 11:36:54.504785 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.504761 2560 scope.go:117] "RemoveContainer" containerID="447a60ab574df3fbca8d6f078dff1c2afa6bd939c5b66da89e9dd7ae28e609fb" Sep 10 11:36:54.513199 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.513180 2560 scope.go:117] "RemoveContainer" containerID="6e603e67f812a27c547ab290a68255e8c74606f8342336813b1df65ab409499f" Sep 10 11:36:54.521802 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.521780 2560 scope.go:117] "RemoveContainer" containerID="29354f5388240e52de43a38fc28e3e7e2134954dd29763c6c11a53ceb9dedbd5" Sep 10 11:36:54.524465 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.524441 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx"] Sep 10 11:36:54.530514 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.530481 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-x56tx"] Sep 10 11:36:54.537183 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.537158 2560 scope.go:117] "RemoveContainer" containerID="11920f556bbf8f9cc82d38c697807c7784eb9cc06e52e8fca081088c37cd1f4e" Sep 10 11:36:54.545703 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.545667 2560 scope.go:117] "RemoveContainer" containerID="e35e04dce4a33b33cd4b0f83c40259bd1d768d924b8336793e3ab6d2f3b38f29" Sep 10 11:36:54.555007 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:54.554985 2560 scope.go:117] "RemoveContainer" containerID="11d33ad769c9475acf0b9b5da24b979d984e9b2ca6595bcefef7e531c28cfa2f" Sep 10 11:36:56.170881 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:36:56.170849 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" path="/var/lib/kubelet/pods/2ef64c0d-633b-4691-afa7-13fe8abac41d/volumes" Sep 10 11:37:06.894728 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:06.894690 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:06.953740 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:06.953703 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:11.968219 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:11.968181 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:12.026083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:12.026048 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:13.027602 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:13.027563 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:13.121504 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:13.121464 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:30.741886 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.741851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:30.873814 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.873779 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-472bx/must-gather-ndw59"] Sep 10 11:37:30.874141 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874130 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="registry" Sep 10 11:37:30.874186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874143 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="registry" Sep 10 11:37:30.874186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874152 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="offline" Sep 10 11:37:30.874186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874158 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="offline" Sep 10 11:37:30.874186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874165 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" Sep 10 11:37:30.874186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874171 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" Sep 10 11:37:30.874186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874179 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" Sep 10 11:37:30.874186 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874185 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874194 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="feast-apply" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874199 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="feast-apply" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874210 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="feast-init" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874215 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="feast-init" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874224 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874230 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874238 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874243 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874254 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="feast-init" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874260 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="feast-init" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874267 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="offline" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874271 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="offline" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874280 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="feast-apply" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874286 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="feast-apply" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874346 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="registry" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874354 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="offline" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874359 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="online" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874365 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="ui" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874371 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef64c0d-633b-4691-afa7-13fe8abac41d" containerName="offline" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874379 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="online" Sep 10 11:37:30.874432 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.874384 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce14a720-757c-4da7-8c26-aca6fe212ce2" containerName="ui" Sep 10 11:37:30.877674 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.877656 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:30.880008 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.879985 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-472bx\"/\"default-dockercfg-qb59r\"" Sep 10 11:37:30.880143 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.880065 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-472bx\"/\"openshift-service-ca.crt\"" Sep 10 11:37:30.880971 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.880952 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-472bx\"/\"kube-root-ca.crt\"" Sep 10 11:37:30.886242 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.886218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:30.886377 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.886225 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-472bx/must-gather-ndw59"] Sep 10 11:37:30.958385 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.958342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lkml\" (UniqueName: \"kubernetes.io/projected/ea521839-d5ff-4698-bfaa-c8274ae7bd3b-kube-api-access-8lkml\") pod \"must-gather-ndw59\" (UID: \"ea521839-d5ff-4698-bfaa-c8274ae7bd3b\") " pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:30.958575 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.958411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ea521839-d5ff-4698-bfaa-c8274ae7bd3b-must-gather-output\") pod \"must-gather-ndw59\" (UID: \"ea521839-d5ff-4698-bfaa-c8274ae7bd3b\") " pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:30.983450 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:30.983417 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:31.059952 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.059853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lkml\" (UniqueName: \"kubernetes.io/projected/ea521839-d5ff-4698-bfaa-c8274ae7bd3b-kube-api-access-8lkml\") pod \"must-gather-ndw59\" (UID: \"ea521839-d5ff-4698-bfaa-c8274ae7bd3b\") " pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:31.059952 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.059911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ea521839-d5ff-4698-bfaa-c8274ae7bd3b-must-gather-output\") pod \"must-gather-ndw59\" (UID: \"ea521839-d5ff-4698-bfaa-c8274ae7bd3b\") " pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:31.060270 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.060249 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ea521839-d5ff-4698-bfaa-c8274ae7bd3b-must-gather-output\") pod \"must-gather-ndw59\" (UID: \"ea521839-d5ff-4698-bfaa-c8274ae7bd3b\") " pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:31.068049 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.068028 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lkml\" (UniqueName: \"kubernetes.io/projected/ea521839-d5ff-4698-bfaa-c8274ae7bd3b-kube-api-access-8lkml\") pod \"must-gather-ndw59\" (UID: \"ea521839-d5ff-4698-bfaa-c8274ae7bd3b\") " pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:31.198423 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.198382 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-472bx/must-gather-ndw59" Sep 10 11:37:31.469170 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.469137 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-472bx/must-gather-ndw59"] Sep 10 11:37:31.471884 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:37:31.471854 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea521839_d5ff_4698_bfaa_c8274ae7bd3b.slice/crio-f0c27d8569fb747a49be6a3141c6acc074ffa29171f43188be7b77abfa7eb7dc WatchSource:0}: Error finding container f0c27d8569fb747a49be6a3141c6acc074ffa29171f43188be7b77abfa7eb7dc: Status 404 returned error can't find the container with id f0c27d8569fb747a49be6a3141c6acc074ffa29171f43188be7b77abfa7eb7dc Sep 10 11:37:31.473774 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.473752 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 11:37:31.655371 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:31.655333 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-472bx/must-gather-ndw59" event={"ID":"ea521839-d5ff-4698-bfaa-c8274ae7bd3b","Type":"ContainerStarted","Data":"f0c27d8569fb747a49be6a3141c6acc074ffa29171f43188be7b77abfa7eb7dc"} Sep 10 11:37:32.662527 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:32.662436 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-472bx/must-gather-ndw59" event={"ID":"ea521839-d5ff-4698-bfaa-c8274ae7bd3b","Type":"ContainerStarted","Data":"5e098117b1ff234251e315252590162483b5b7ae555e582d09589cc2fbeaf7e0"} Sep 10 11:37:32.662527 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:32.662480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-472bx/must-gather-ndw59" event={"ID":"ea521839-d5ff-4698-bfaa-c8274ae7bd3b","Type":"ContainerStarted","Data":"a23e5d057c844fa39df160d681a157f143ff395f8620c8959123989693791f97"} Sep 10 11:37:32.679585 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:32.679519 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-472bx/must-gather-ndw59" podStartSLOduration=1.898174429 podStartE2EDuration="2.679495679s" totalImagesPullingTime="781.32125ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:37:32.678490251 +0000 UTC m=+1491.037546457" watchObservedRunningTime="2026-09-10 11:37:32.679495679 +0000 UTC m=+1491.038551886" Sep 10 11:37:33.725434 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:33.725389 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dsmzj_7f5e0a42-7fba-460a-b3d2-67a92ca2c94e/global-pull-secret-syncer/0.log" Sep 10 11:37:33.816187 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:33.816132 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" Sep 10 11:37:33.903467 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:33.903439 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-79.ec2.internal_fe6dc34d7642815f28f29ef9663833c3/haproxy/0.log" Sep 10 11:37:36.395726 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:36.395683 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:36.496600 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:36.496509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ppb4s_aecc4dcd-2c08-453f-8202-1f571e23862a/konnectivity-agent/0.log" limit=1024 Sep 10 11:37:37.599742 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.599717 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a8c02dba-388a-47c1-ae01-38bd1da194fd/alertmanager/0.log" Sep 10 11:37:37.629318 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.629269 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a8c02dba-388a-47c1-ae01-38bd1da194fd/config-reloader/0.log" Sep 10 11:37:37.661164 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.661135 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a8c02dba-388a-47c1-ae01-38bd1da194fd/kube-rbac-proxy-web/0.log" Sep 10 11:37:37.687873 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.687845 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a8c02dba-388a-47c1-ae01-38bd1da194fd/kube-rbac-proxy/0.log" Sep 10 11:37:37.716351 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.716322 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a8c02dba-388a-47c1-ae01-38bd1da194fd/kube-rbac-proxy-metric/0.log" Sep 10 11:37:37.745046 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.745016 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a8c02dba-388a-47c1-ae01-38bd1da194fd/prom-label-proxy/0.log" Sep 10 11:37:37.772561 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.772526 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a8c02dba-388a-47c1-ae01-38bd1da194fd/init-config-reloader/0.log" Sep 10 11:37:37.847212 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.847178 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-x84rv_acbc0d41-f803-4bde-9b12-3042136b601e/kube-state-metrics/0.log" Sep 10 11:37:37.870499 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.870421 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-x84rv_acbc0d41-f803-4bde-9b12-3042136b601e/kube-rbac-proxy-main/0.log" Sep 10 11:37:37.908685 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.908649 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-x84rv_acbc0d41-f803-4bde-9b12-3042136b601e/kube-rbac-proxy-self/0.log" Sep 10 11:37:37.941146 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.941115 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-ff66b8fb-9xtkv_b3bc84d1-9c77-4f59-9674-2439981a24af/metrics-server/0.log" Sep 10 11:37:37.974385 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:37.974350 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-72m69_21d22bca-990d-44c3-967d-1989bf231b5f/monitoring-plugin/0.log" Sep 10 11:37:38.135846 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.135814 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5xtzc_7d8d509c-426a-4ac0-a3dd-90c2756bff1f/node-exporter/0.log" Sep 10 11:37:38.163345 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.163313 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5xtzc_7d8d509c-426a-4ac0-a3dd-90c2756bff1f/kube-rbac-proxy/0.log" Sep 10 11:37:38.202687 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.202658 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5xtzc_7d8d509c-426a-4ac0-a3dd-90c2756bff1f/init-textfile/0.log" Sep 10 11:37:38.316667 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.316625 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-459rp_16196610-c43e-4d83-a1e5-06d39a0e3bfc/kube-rbac-proxy-main/0.log" Sep 10 11:37:38.340848 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.340742 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-459rp_16196610-c43e-4d83-a1e5-06d39a0e3bfc/kube-rbac-proxy-self/0.log" Sep 10 11:37:38.368095 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.368061 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-459rp_16196610-c43e-4d83-a1e5-06d39a0e3bfc/openshift-state-metrics/0.log" Sep 10 11:37:38.410216 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.410118 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bf395214-0110-4f09-9feb-f90dd2fe02d1/prometheus/0.log" Sep 10 11:37:38.433075 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.433048 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bf395214-0110-4f09-9feb-f90dd2fe02d1/config-reloader/0.log" Sep 10 11:37:38.456052 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.456022 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bf395214-0110-4f09-9feb-f90dd2fe02d1/thanos-sidecar/0.log" Sep 10 11:37:38.482486 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.482431 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bf395214-0110-4f09-9feb-f90dd2fe02d1/kube-rbac-proxy-web/0.log" Sep 10 11:37:38.506938 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.506902 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bf395214-0110-4f09-9feb-f90dd2fe02d1/kube-rbac-proxy/0.log" Sep 10 11:37:38.532913 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.532876 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bf395214-0110-4f09-9feb-f90dd2fe02d1/kube-rbac-proxy-thanos/0.log" Sep 10 11:37:38.562149 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.562120 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bf395214-0110-4f09-9feb-f90dd2fe02d1/init-config-reloader/0.log" Sep 10 11:37:38.602046 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.601972 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-8rgk6_bf8f48fd-100a-4ce0-ba11-03c9cbe056ba/prometheus-operator/0.log" Sep 10 11:37:38.624761 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.624730 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-8rgk6_bf8f48fd-100a-4ce0-ba11-03c9cbe056ba/kube-rbac-proxy/0.log" Sep 10 11:37:38.684671 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.684634 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d45487466-wxmzn_a49020fd-3607-4d2c-80ea-7fa2315648af/telemeter-client/0.log" Sep 10 11:37:38.712836 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.712801 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d45487466-wxmzn_a49020fd-3607-4d2c-80ea-7fa2315648af/reload/0.log" Sep 10 11:37:38.739942 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.739908 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d45487466-wxmzn_a49020fd-3607-4d2c-80ea-7fa2315648af/kube-rbac-proxy/0.log" Sep 10 11:37:38.784892 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.784854 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74f774c495-5crkk_eb05e7bf-65d9-4b36-a486-34294952a59c/thanos-query/0.log" Sep 10 11:37:38.814174 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.814139 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74f774c495-5crkk_eb05e7bf-65d9-4b36-a486-34294952a59c/kube-rbac-proxy-web/0.log" Sep 10 11:37:38.838966 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.838917 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74f774c495-5crkk_eb05e7bf-65d9-4b36-a486-34294952a59c/kube-rbac-proxy/0.log" Sep 10 11:37:38.871148 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.871114 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74f774c495-5crkk_eb05e7bf-65d9-4b36-a486-34294952a59c/prom-label-proxy/0.log" Sep 10 11:37:38.902142 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.902112 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74f774c495-5crkk_eb05e7bf-65d9-4b36-a486-34294952a59c/kube-rbac-proxy-rules/0.log" Sep 10 11:37:38.933936 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:38.933893 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74f774c495-5crkk_eb05e7bf-65d9-4b36-a486-34294952a59c/kube-rbac-proxy-metrics/0.log" Sep 10 11:37:41.092523 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.092493 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6699dbc47b-gf8xr_d211812c-aaaa-4c9e-aca3-c3959f5a9bb7/console/0.log" Sep 10 11:37:41.567784 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.567744 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh"] Sep 10 11:37:41.573425 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.573393 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.582334 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.582271 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh"] Sep 10 11:37:41.667752 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.667715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-lib-modules\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.667942 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.667772 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-sys\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.667942 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.667820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-podres\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.667942 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.667929 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfnc5\" (UniqueName: \"kubernetes.io/projected/3c5200d0-8862-4e1d-820f-6914df87a23f-kube-api-access-vfnc5\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.668096 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.667958 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-proc\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.768824 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.768784 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-lib-modules\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.768824 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.768830 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-sys\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.769083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.768857 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-podres\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.769083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.768913 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfnc5\" (UniqueName: \"kubernetes.io/projected/3c5200d0-8862-4e1d-820f-6914df87a23f-kube-api-access-vfnc5\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.769083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.768934 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-proc\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.769083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.768932 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-sys\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.769083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.768999 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-lib-modules\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.769083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.769012 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-proc\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.769083 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.769018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c5200d0-8862-4e1d-820f-6914df87a23f-podres\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.779278 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.779235 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfnc5\" (UniqueName: \"kubernetes.io/projected/3c5200d0-8862-4e1d-820f-6914df87a23f-kube-api-access-vfnc5\") pod \"perf-node-gather-daemonset-tbwfh\" (UID: \"3c5200d0-8862-4e1d-820f-6914df87a23f\") " pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:41.887258 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:41.887166 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:42.173025 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.172860 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh"] Sep 10 11:37:42.176005 ip-10-0-131-79 kubenswrapper[2560]: W0910 11:37:42.175972 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3c5200d0_8862_4e1d_820f_6914df87a23f.slice/crio-a9780314828c5747ddbeb62e3f391ec917a612aa45cc0f8d641f75effb614ea5 WatchSource:0}: Error finding container a9780314828c5747ddbeb62e3f391ec917a612aa45cc0f8d641f75effb614ea5: Status 404 returned error can't find the container with id a9780314828c5747ddbeb62e3f391ec917a612aa45cc0f8d641f75effb614ea5 Sep 10 11:37:42.187055 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.187036 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:37:42.189799 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.189773 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:37:42.529707 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.529630 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wdg7_564e65e3-d691-42fe-afa5-2c86e533fc6f/dns/0.log" Sep 10 11:37:42.553555 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.553523 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9wdg7_564e65e3-d691-42fe-afa5-2c86e533fc6f/kube-rbac-proxy/0.log" Sep 10 11:37:42.703316 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.703268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pqv7s_b2b64fd3-a514-4a75-acec-78a1678d4044/dns-node-resolver/0.log" Sep 10 11:37:42.719824 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.719782 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" event={"ID":"3c5200d0-8862-4e1d-820f-6914df87a23f","Type":"ContainerStarted","Data":"d5207c571cb9ed2f483409c61b2c4ba53785e922917be91a019b55128b476b0e"} Sep 10 11:37:42.720028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.719888 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" event={"ID":"3c5200d0-8862-4e1d-820f-6914df87a23f","Type":"ContainerStarted","Data":"a9780314828c5747ddbeb62e3f391ec917a612aa45cc0f8d641f75effb614ea5"} Sep 10 11:37:42.720164 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.720136 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:42.737957 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:42.737902 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" podStartSLOduration=1.7378865270000001 podStartE2EDuration="1.737886527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:37:42.736059629 +0000 UTC m=+1501.095115836" watchObservedRunningTime="2026-09-10 11:37:42.737886527 +0000 UTC m=+1501.096942733" Sep 10 11:37:43.217034 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:43.217004 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mt6pz_a1b5e602-0217-49c5-8c46-502120dff851/node-ca/0.log" Sep 10 11:37:44.266750 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:44.266714 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f5dhc_3b93c15f-256d-4f14-b104-cc20cadc496c/serve-healthcheck-canary/0.log" Sep 10 11:37:44.728459 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:44.728429 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkmkp_e718f029-cbb3-4784-bb09-bb828ec8ee3b/kube-rbac-proxy/0.log" Sep 10 11:37:44.749132 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:44.749106 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkmkp_e718f029-cbb3-4784-bb09-bb828ec8ee3b/exporter/0.log" Sep 10 11:37:44.771850 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:44.771820 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkmkp_e718f029-cbb3-4784-bb09-bb828ec8ee3b/extractor/0.log" Sep 10 11:37:48.736225 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:48.736199 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-472bx/perf-node-gather-daemonset-tbwfh" Sep 10 11:37:50.463516 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.463480 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-52mbp_bc0914eb-81b1-41b1-8553-83522073557c/kube-multus/0.log" Sep 10 11:37:50.525684 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.525652 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-244zd_eff84843-6d59-4eaf-ac58-1740f3ce992f/kube-multus-additional-cni-plugins/0.log" Sep 10 11:37:50.553734 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.553652 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-244zd_eff84843-6d59-4eaf-ac58-1740f3ce992f/egress-router-binary-copy/0.log" Sep 10 11:37:50.579311 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.579268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-244zd_eff84843-6d59-4eaf-ac58-1740f3ce992f/cni-plugins/0.log" Sep 10 11:37:50.613739 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.613710 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-244zd_eff84843-6d59-4eaf-ac58-1740f3ce992f/bond-cni-plugin/0.log" Sep 10 11:37:50.652046 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.652010 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-244zd_eff84843-6d59-4eaf-ac58-1740f3ce992f/routeoverride-cni/0.log" Sep 10 11:37:50.702716 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.702680 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-244zd_eff84843-6d59-4eaf-ac58-1740f3ce992f/whereabouts-cni-bincopy/0.log" Sep 10 11:37:50.729745 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:50.729715 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-244zd_eff84843-6d59-4eaf-ac58-1740f3ce992f/whereabouts-cni/0.log" Sep 10 11:37:51.245997 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:51.245885 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r42wc_f4f8581c-3527-40bb-89f6-b51b45a0d4af/network-metrics-daemon/0.log" Sep 10 11:37:51.270212 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:51.270181 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r42wc_f4f8581c-3527-40bb-89f6-b51b45a0d4af/kube-rbac-proxy/0.log" Sep 10 11:37:52.436968 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.436897 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-controller/0.log" Sep 10 11:37:52.455792 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.455759 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/0.log" Sep 10 11:37:52.455961 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.455766 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovn-acl-logging/1.log" Sep 10 11:37:52.483974 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.483945 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/kube-rbac-proxy-node/0.log" Sep 10 11:37:52.507927 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.507896 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 11:37:52.530247 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.530222 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/northd/0.log" Sep 10 11:37:52.562806 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.562779 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/nbdb/0.log" Sep 10 11:37:52.587028 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.586998 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/sbdb/0.log" Sep 10 11:37:52.631839 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:52.631811 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v4578_811e7c53-3cb8-4df3-96d1-e1e823aa63ee/ovnkube-controller/0.log" Sep 10 11:37:53.654646 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:53.654613 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-9tdqv_6ca7ad53-128f-4ce9-8dc7-ffa5482323c8/network-check-target-container/0.log" Sep 10 11:37:54.758697 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:54.758666 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-hf4jq_a3997314-3e5b-49cf-b445-291aad983581/iptables-alerter/0.log" Sep 10 11:37:55.487583 ip-10-0-131-79 kubenswrapper[2560]: I0910 11:37:55.487556 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-4qq68_70b2b1a4-e63a-4fb3-aa44-ebc01c2e103b/tuned/0.log"