Sep 09 20:59:29.815679 ip-10-0-133-18 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 20:59:29.815690 ip-10-0-133-18 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 20:59:29.815697 ip-10-0-133-18 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 20:59:29.816052 ip-10-0-133-18 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 20:59:39.848541 ip-10-0-133-18 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 20:59:39.848558 ip-10-0-133-18 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3812c1d9c6434156a806c0a64012fc1e -- Sep 09 21:01:37.072497 ip-10-0-133-18 systemd[1]: Starting Kubernetes Kubelet... Sep 09 21:01:37.463350 ip-10-0-133-18 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:01:37.463350 ip-10-0-133-18 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 21:01:37.463350 ip-10-0-133-18 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:01:37.463350 ip-10-0-133-18 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 21:01:37.463350 ip-10-0-133-18 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:01:37.464928 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.464822 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 21:01:37.472514 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472494 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:37.472514 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472511 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:37.472514 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472515 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:37.472514 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472518 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:37.472514 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472521 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472524 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472527 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472529 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472533 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472535 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472538 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472541 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472545 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472549 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472551 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472554 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472557 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472565 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472568 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472571 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472573 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472576 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472579 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472581 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:37.472698 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472584 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472586 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472589 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472591 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472596 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472599 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472602 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472604 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472607 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472610 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472612 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472614 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472616 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472620 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472623 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472625 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472628 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472631 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472633 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:37.473164 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472636 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472638 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472641 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472643 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472646 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472648 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472651 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472659 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472661 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472664 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472666 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472668 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472672 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472674 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472676 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472679 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472681 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472684 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472686 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472688 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:37.473669 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472691 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472694 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472696 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472698 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472701 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472704 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472707 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472710 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472713 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472715 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472718 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472720 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472723 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472725 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472728 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472730 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472733 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472737 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472740 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472743 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:37.474141 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472751 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472754 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472757 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472760 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472762 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472765 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472768 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472770 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472773 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472776 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472778 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472781 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472783 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472786 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472788 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.472791 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473550 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473557 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473561 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473565 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:37.474641 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473568 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473571 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473574 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473576 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473579 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473583 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473587 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473590 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473593 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473595 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473598 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473600 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473603 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473611 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473614 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473616 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473619 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473621 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473624 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473626 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:37.475110 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473629 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473631 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473635 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473639 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473653 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473656 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473659 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473662 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473665 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473667 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473671 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473675 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473678 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473680 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473683 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473686 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473688 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473691 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473693 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:37.475628 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473696 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473698 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473701 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473703 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473706 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473708 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473711 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473718 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473721 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473723 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473726 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473728 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473731 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473733 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473736 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473738 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473741 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473743 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473746 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473748 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:37.476099 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473751 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473753 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473757 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473759 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473762 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473765 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473767 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473770 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473772 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473774 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473776 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473779 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473782 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473785 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473787 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473790 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473792 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473795 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473797 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473800 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:37.476608 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473808 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473810 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473814 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473817 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473820 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473823 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473826 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473829 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473832 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473834 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473836 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473839 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473841 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473844 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473846 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.473848 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474270 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474276 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474279 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:37.477101 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474282 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474285 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474288 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474291 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474293 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474296 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474299 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474302 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474304 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474307 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474310 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474312 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474315 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474318 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474320 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474323 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474325 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474328 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474330 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474332 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:37.477591 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474335 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474338 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474340 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474344 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474346 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474348 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474351 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474353 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474356 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474360 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474363 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474365 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474368 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474370 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474373 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474375 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474379 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474383 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474386 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474388 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:37.478075 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474391 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474394 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474397 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474399 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474402 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474406 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474409 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474413 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474415 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474418 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474421 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474423 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474426 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474428 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474430 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474433 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474436 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474438 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474441 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474443 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:37.478612 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474446 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474449 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474452 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474455 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474457 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474460 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474462 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474465 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474469 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474472 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474474 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474477 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474480 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474483 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474486 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474488 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474491 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474495 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474498 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474501 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:37.479104 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474503 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474507 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474510 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474512 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474515 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474517 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474520 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474523 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474527 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474529 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474532 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474534 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474537 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474541 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474544 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.474546 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474615 2578 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474650 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474657 2578 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474661 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 21:01:37.479600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474665 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474668 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474672 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474676 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474679 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474682 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474685 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474689 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474691 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474696 2578 flags.go:64] FLAG: --cgroup-root="" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474699 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474702 2578 flags.go:64] FLAG: --client-ca-file="" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474705 2578 flags.go:64] FLAG: --cloud-provider="external" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474708 2578 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474712 2578 flags.go:64] FLAG: --cluster-domain="" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474715 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474718 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474720 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474723 2578 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474727 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474730 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474732 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474736 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474738 2578 flags.go:64] FLAG: --contention-profiling="false" Sep 09 21:01:37.480076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474741 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474745 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474748 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474751 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474755 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474758 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474760 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474763 2578 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474766 2578 flags.go:64] FLAG: --enable-server="true" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474768 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474772 2578 flags.go:64] FLAG: --event-burst="100" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474775 2578 flags.go:64] FLAG: --event-qps="50" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474778 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474781 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474784 2578 flags.go:64] FLAG: --eviction-hard="" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474788 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474790 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474794 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474797 2578 flags.go:64] FLAG: --eviction-soft="" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474800 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474802 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474806 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474809 2578 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474811 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474814 2578 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 21:01:37.480665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474817 2578 flags.go:64] FLAG: --feature-gates="" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474820 2578 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474823 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474826 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474829 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474831 2578 flags.go:64] FLAG: --healthz-port="10248" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474834 2578 flags.go:64] FLAG: --help="false" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474837 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474842 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474845 2578 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474847 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474851 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474854 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474856 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474859 2578 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474861 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474864 2578 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474867 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474870 2578 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474872 2578 flags.go:64] FLAG: --kube-reserved="" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474875 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474878 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474881 2578 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474883 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 21:01:37.481289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474887 2578 flags.go:64] FLAG: --lock-file="" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474890 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474892 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474895 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474900 2578 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474904 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474906 2578 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474909 2578 flags.go:64] FLAG: --logging-format="text" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474911 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474915 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474917 2578 flags.go:64] FLAG: --manifest-url="" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474920 2578 flags.go:64] FLAG: --manifest-url-header="" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474923 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474926 2578 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474930 2578 flags.go:64] FLAG: --max-pods="110" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474933 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474937 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474939 2578 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474942 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474945 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474948 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474951 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474957 2578 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474960 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474963 2578 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 21:01:37.481949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474966 2578 flags.go:64] FLAG: --pod-cidr="" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474969 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474974 2578 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474977 2578 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474980 2578 flags.go:64] FLAG: --pods-per-core="0" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474983 2578 flags.go:64] FLAG: --port="10250" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474986 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474990 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0697257c0f773b33d" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474993 2578 flags.go:64] FLAG: --qos-reserved="" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474996 2578 flags.go:64] FLAG: --read-only-port="10255" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.474999 2578 flags.go:64] FLAG: --register-node="true" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475001 2578 flags.go:64] FLAG: --register-with-taints="" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475005 2578 flags.go:64] FLAG: --registry-burst="10" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475008 2578 flags.go:64] FLAG: --registry-qps="5" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475011 2578 flags.go:64] FLAG: --reserved-cpus="" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475014 2578 flags.go:64] FLAG: --reserved-memory="" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475017 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475020 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475023 2578 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475025 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475028 2578 flags.go:64] FLAG: --runonce="false" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475031 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475033 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475036 2578 flags.go:64] FLAG: --seccomp-default="false" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475040 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 21:01:37.482569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475043 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475046 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475049 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475051 2578 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475054 2578 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475057 2578 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475059 2578 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475062 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475065 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475068 2578 flags.go:64] FLAG: --system-cgroups="" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475070 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475075 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475078 2578 flags.go:64] FLAG: --tls-cert-file="" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475080 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475085 2578 flags.go:64] FLAG: --tls-min-version="" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475088 2578 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475090 2578 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475093 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475096 2578 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475099 2578 flags.go:64] FLAG: --v="2" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475102 2578 flags.go:64] FLAG: --version="false" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475107 2578 flags.go:64] FLAG: --vmodule="" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475110 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475114 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475249 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:37.483271 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475254 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475257 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475260 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475263 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475265 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475268 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475274 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475277 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475279 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475282 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475285 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475287 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475290 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475293 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475295 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475298 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475301 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475303 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475306 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475308 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:37.483899 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475310 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475315 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475317 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475320 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475322 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475325 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475327 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475330 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475332 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475335 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475338 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475340 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475343 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475347 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475350 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475353 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475355 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475357 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475361 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:37.484420 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475364 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475366 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475368 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475371 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475375 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475378 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475381 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475384 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475387 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475390 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475393 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475396 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475399 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475402 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475406 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475409 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475411 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475414 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475417 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475419 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:37.484901 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475422 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475425 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475427 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475430 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475433 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475435 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475438 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475440 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475443 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475445 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475448 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475452 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475454 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475457 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475459 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475462 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475464 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475467 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475469 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475472 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:37.485447 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475474 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475478 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475481 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475484 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475487 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475489 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475493 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475496 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475498 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475501 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475504 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475506 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475509 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475511 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475514 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475516 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475519 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475528 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:37.485934 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.475531 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.475537 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.483121 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.483135 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.483732 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.483739 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483801 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483806 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483809 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483813 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483815 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483818 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483821 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483824 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:37.486464 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483826 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483831 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483834 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483838 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483841 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483843 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483847 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483852 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483854 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483857 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483860 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483862 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483865 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483868 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483870 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483873 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483883 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483886 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483888 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:37.486858 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483891 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483894 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483897 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483900 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483903 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483906 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483908 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483911 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483913 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483916 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483918 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483921 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483923 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483926 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483928 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483931 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483933 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483936 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483938 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483941 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:37.487327 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483943 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483946 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483948 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483951 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483955 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483958 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483961 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483964 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483967 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483970 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483976 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483979 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483982 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483985 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483988 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483991 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483994 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.483997 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484000 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484003 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:37.487846 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484005 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484008 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484011 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484013 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484016 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484018 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484020 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484023 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484026 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484028 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484030 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484033 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484035 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484038 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484040 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484043 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484045 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484048 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484051 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484053 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:37.488350 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484055 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484058 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484060 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484068 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484071 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484073 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484076 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484078 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484081 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484083 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484086 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484088 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.484093 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484226 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:01:37.488832 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484232 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484235 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484238 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484241 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484243 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484246 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484248 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484251 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484253 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484256 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484258 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484261 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484263 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484265 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484269 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484272 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484275 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484277 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484280 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:01:37.489175 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484282 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484286 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484289 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484298 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484301 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484304 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484307 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484310 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484313 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484315 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484318 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484321 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484324 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484327 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484329 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484332 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484334 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484337 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484339 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484341 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:01:37.489651 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484344 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484348 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484351 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484354 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484357 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484360 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484362 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484365 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484368 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484370 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484373 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484375 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484378 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484380 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484383 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484385 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484388 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484391 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484393 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:01:37.490151 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484396 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484399 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484401 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484404 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484406 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484409 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484411 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484414 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484416 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484419 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484421 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484424 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484427 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484429 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484432 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484435 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484437 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484440 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484442 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484445 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:01:37.490654 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484447 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484449 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484452 2578 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484454 2578 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484457 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484459 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484461 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484464 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484466 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484469 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484471 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484474 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484476 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484479 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484481 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484483 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484486 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484488 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484491 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:01:37.491150 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:37.484493 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:01:37.491627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.484498 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:01:37.491627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.484596 2578 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 21:01:37.491627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.486462 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 21:01:37.491627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.487378 2578 server.go:1019] "Starting client certificate rotation" Sep 09 21:01:37.491627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.487480 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:01:37.491627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.487528 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:01:37.491627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.489733 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:01:37.494338 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.494323 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:01:37.505777 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.505755 2578 log.go:25] "Validated CRI v1 runtime API" Sep 09 21:01:37.508551 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.508537 2578 log.go:25] "Validated CRI v1 image API" Sep 09 21:01:37.511210 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.511181 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 21:01:37.514187 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.514168 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b4384637-19cd-4d46-9fce-f3fe0ad01e69:/dev/nvme0n1p4 bf708103-0dc1-48a0-9499-2ead4cbbe96a:/dev/nvme0n1p3] Sep 09 21:01:37.514266 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.514186 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 21:01:37.516053 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.516037 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:01:37.519178 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.519067 2578 manager.go:216] Machine: {Timestamp:2026-09-09 21:01:37.517793565 +0000 UTC m=+0.331643705 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104897 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2311cf50bd14b4e2be3563ff11a830 SystemUUID:ec2311cf-50bd-14b4-e2be-3563ff11a830 BootID:3812c1d9-c643-4156-a806-c0a64012fc1e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:31:2c:b2:19:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:31:2c:b2:19:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:2b:82:44:a1:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 21:01:37.519178 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.519173 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 21:01:37.519286 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.519259 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 21:01:37.522948 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.522831 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 21:01:37.523078 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.522952 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-18.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 21:01:37.523655 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.523645 2578 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 21:01:37.523690 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.523657 2578 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 21:01:37.523690 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.523666 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:01:37.524322 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.524312 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:01:37.525138 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.525129 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 21:01:37.528029 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528017 2578 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:01:37.528147 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528137 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 21:01:37.528176 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528166 2578 kubelet.go:486] "Attempting to sync node with API server" Sep 09 21:01:37.528216 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528176 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 21:01:37.528216 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528190 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 21:01:37.528309 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528217 2578 kubelet.go:398] "Adding apiserver pod source" Sep 09 21:01:37.528309 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528228 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 21:01:37.528612 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528593 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:01:37.528696 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.528620 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:01:37.530787 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.530767 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 21:01:37.532883 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.532870 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 21:01:37.532931 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.532901 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 21:01:37.534668 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534654 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 21:01:37.534707 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534674 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 21:01:37.534707 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534680 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 21:01:37.534707 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534686 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 21:01:37.534707 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534694 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 21:01:37.534814 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534709 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 21:01:37.534814 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534722 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 21:01:37.534814 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534727 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 21:01:37.534814 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534733 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 21:01:37.534814 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534738 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 21:01:37.534814 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534750 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 21:01:37.534814 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.534755 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 21:01:37.536252 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.536242 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 21:01:37.536252 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.536253 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 21:01:37.537881 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.537855 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 21:01:37.537881 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.537852 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 21:01:37.540755 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.540739 2578 server.go:1268] "Started kubelet" Sep 09 21:01:37.541037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.541009 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 21:01:37.541185 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.541162 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-18.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 21:01:37.541322 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.541253 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 21:01:37.541387 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.541357 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 21:01:37.541488 ip-10-0-133-18 systemd[1]: Started Kubernetes Kubelet. Sep 09 21:01:37.542212 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.542183 2578 server.go:310] "Adding debug handlers to kubelet server" Sep 09 21:01:37.542511 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.542343 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 21:01:37.545609 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.545591 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 21:01:37.547510 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.546540 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51b751182 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.540698498 +0000 UTC m=+0.354548637,LastTimestamp:2026-09-09 21:01:37.540698498 +0000 UTC m=+0.354548637,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.548191 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.548174 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 21:01:37.548996 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.548978 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 21:01:37.549100 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.549085 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:37.549768 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.549754 2578 factory.go:55] Registering systemd factory Sep 09 21:01:37.549768 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.549769 2578 factory.go:223] Registration of the systemd container factory successfully Sep 09 21:01:37.549955 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.549938 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 21:01:37.549955 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.549939 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 21:01:37.550063 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.549958 2578 factory.go:153] Registering CRI-O factory Sep 09 21:01:37.550063 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.549965 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 21:01:37.550063 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.549966 2578 factory.go:223] Registration of the crio container factory successfully Sep 09 21:01:37.550063 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.550042 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 21:01:37.550063 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.550062 2578 factory.go:103] Registering Raw factory Sep 09 21:01:37.550328 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.550068 2578 reconstruct.go:97] "Volume reconstruction finished" Sep 09 21:01:37.550328 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.550073 2578 manager.go:1196] Started watching for new ooms in manager Sep 09 21:01:37.550328 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.550076 2578 reconciler.go:29] "Reconciler: start to sync state" Sep 09 21:01:37.550470 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.550436 2578 manager.go:349] Starting recovery of all containers Sep 09 21:01:37.553288 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.553183 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 21:01:37.553355 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.553306 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 21:01:37.557919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.557878 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 21:01:37.560932 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.560909 2578 manager.go:354] Recovery completed Sep 09 21:01:37.566431 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.566416 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:37.568001 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.567986 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:37.568067 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.568011 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:37.568067 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.568022 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:37.568605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.568592 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 21:01:37.568605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.568605 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 21:01:37.568696 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.568619 2578 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:01:37.568696 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.568644 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:37.569289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.569273 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:37.569356 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.569302 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:37.569356 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.569315 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:37.569834 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.569821 2578 policy_none.go:49] "None policy: Start" Sep 09 21:01:37.569867 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.569839 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 21:01:37.569867 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.569848 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 21:01:37.570661 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.570594 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51d15a7eb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-18.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.567999979 +0000 UTC m=+0.381850119,LastTimestamp:2026-09-09 21:01:37.567999979 +0000 UTC m=+0.381850119,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.570978 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.570967 2578 policy_none.go:47] "Start" Sep 09 21:01:37.579637 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.579578 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51d15e6ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-18.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.568016078 +0000 UTC m=+0.381866218,LastTimestamp:2026-09-09 21:01:37.568016078 +0000 UTC m=+0.381866218,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.590696 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.590641 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51d160bb5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-18.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.568025525 +0000 UTC m=+0.381875665,LastTimestamp:2026-09-09 21:01:37.568025525 +0000 UTC m=+0.381875665,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.601375 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.601177 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-18.ec2.internal.18d3c2d51d15a7eb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51d15a7eb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-18.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.567999979 +0000 UTC m=+0.381850119,LastTimestamp:2026-09-09 21:01:37.56928821 +0000 UTC m=+0.383138350,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.607767 2578 manager.go:337] "Starting Device Plugin manager" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.607798 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.607807 2578 server.go:87] "Starting device plugin registration server" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.608095 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.608111 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.608249 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.608366 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.608376 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.608796 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.608840 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:37.619068 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.610537 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-18.ec2.internal.18d3c2d51d15e6ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51d15e6ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-18.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.568016078 +0000 UTC m=+0.381866218,LastTimestamp:2026-09-09 21:01:37.569309557 +0000 UTC m=+0.383159697,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.619451 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.619093 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-18.ec2.internal.18d3c2d51d160bb5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51d160bb5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-18.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.568025525 +0000 UTC m=+0.381875665,LastTimestamp:2026-09-09 21:01:37.569318924 +0000 UTC m=+0.383169064,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.630070 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.630017 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-18.ec2.internal.18d3c2d51f96c85d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-18.ec2.internal,UID:ip-10-0-133-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-18.ec2.internal,},FirstTimestamp:2026-09-09 21:01:37.610016861 +0000 UTC m=+0.423866991,LastTimestamp:2026-09-09 21:01:37.610016861 +0000 UTC m=+0.423866991,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-18.ec2.internal,}" Sep 09 21:01:37.659796 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.659775 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tzbls" Sep 09 21:01:37.675589 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.675572 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tzbls" Sep 09 21:01:37.681019 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.681004 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 21:01:37.681097 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.681050 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 21:01:37.681097 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.681069 2578 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 21:01:37.681159 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.681101 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 21:01:37.690580 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.690560 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:37.708484 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.708469 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:37.709152 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.709138 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:37.709221 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.709169 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:37.709221 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.709179 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:37.709301 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.709234 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.722731 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.722679 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.722731 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.722699 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-18.ec2.internal\": node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:37.748592 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.748575 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:37.781559 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.781531 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal"] Sep 09 21:01:37.781634 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.781599 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:37.782866 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.782852 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:37.782958 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.782883 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:37.782958 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.782900 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:37.784135 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784121 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:37.784271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784243 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.784348 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784284 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:37.784805 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784786 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:37.784805 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784799 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:37.784902 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784819 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:37.784902 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784832 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:37.784902 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784821 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:37.784902 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.784893 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:37.785969 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.785957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.786012 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.785980 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:37.786597 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.786583 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:37.786686 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.786609 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:37.786686 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.786620 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:37.810869 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.810847 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-18.ec2.internal\" not found" node="ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.815138 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.815125 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-18.ec2.internal\" not found" node="ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.849250 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.849232 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:37.850987 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.850965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/548898899e6221ee528d5a8d7f51c774-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal\" (UID: \"548898899e6221ee528d5a8d7f51c774\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.851072 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.850989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548898899e6221ee528d5a8d7f51c774-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal\" (UID: \"548898899e6221ee528d5a8d7f51c774\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.851072 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.851006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0559b432e0a8e8090650f241aa697b78-config\") pod \"kube-apiserver-proxy-ip-10-0-133-18.ec2.internal\" (UID: \"0559b432e0a8e8090650f241aa697b78\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.949345 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:37.949327 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:37.951497 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.951484 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/548898899e6221ee528d5a8d7f51c774-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal\" (UID: \"548898899e6221ee528d5a8d7f51c774\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.951557 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.951505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548898899e6221ee528d5a8d7f51c774-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal\" (UID: \"548898899e6221ee528d5a8d7f51c774\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.951557 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.951533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0559b432e0a8e8090650f241aa697b78-config\") pod \"kube-apiserver-proxy-ip-10-0-133-18.ec2.internal\" (UID: \"0559b432e0a8e8090650f241aa697b78\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.951627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.951574 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548898899e6221ee528d5a8d7f51c774-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal\" (UID: \"548898899e6221ee528d5a8d7f51c774\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.951627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.951588 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0559b432e0a8e8090650f241aa697b78-config\") pod \"kube-apiserver-proxy-ip-10-0-133-18.ec2.internal\" (UID: \"0559b432e0a8e8090650f241aa697b78\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" Sep 09 21:01:37.951627 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:37.951574 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/548898899e6221ee528d5a8d7f51c774-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal\" (UID: \"548898899e6221ee528d5a8d7f51c774\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:38.049958 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.049897 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.113204 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.113173 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:38.118096 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.118080 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" Sep 09 21:01:38.149990 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.149966 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.250308 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.250283 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.350769 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.350710 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.451239 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.451214 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.452428 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.452414 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:38.487263 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.487240 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 21:01:38.487679 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.487391 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 21:01:38.487679 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.487409 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 21:01:38.552023 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.551996 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.652365 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.652305 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.679607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.679579 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:56:38 +0000 UTC" deadline="2028-06-12 23:45:16.008960576 +0000 UTC" Sep 09 21:01:38.679607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.679606 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15410h43m37.329357684s" Sep 09 21:01:38.696959 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.696934 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:38.724772 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.724751 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:01:38.725983 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.725967 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:01:38.726060 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.726002 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:01:38.726060 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.726014 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:01:38.752634 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.752611 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.816391 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:38.816362 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0559b432e0a8e8090650f241aa697b78.slice/crio-6f4b1c14a4b919945745772743df2afc720ee58fd5e87eee1a02b1e87db2b611 WatchSource:0}: Error finding container 6f4b1c14a4b919945745772743df2afc720ee58fd5e87eee1a02b1e87db2b611: Status 404 returned error can't find the container with id 6f4b1c14a4b919945745772743df2afc720ee58fd5e87eee1a02b1e87db2b611 Sep 09 21:01:38.816768 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:38.816753 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod548898899e6221ee528d5a8d7f51c774.slice/crio-a4516f7191cc513b7d5540581a7efb2f009cd973ad7f957fa82605e04ab3d531 WatchSource:0}: Error finding container a4516f7191cc513b7d5540581a7efb2f009cd973ad7f957fa82605e04ab3d531: Status 404 returned error can't find the container with id a4516f7191cc513b7d5540581a7efb2f009cd973ad7f957fa82605e04ab3d531 Sep 09 21:01:38.822082 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:38.822068 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:01:38.853641 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.853623 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:38.954091 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:38.954070 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:39.054396 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.054350 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-18.ec2.internal\" not found" Sep 09 21:01:39.063317 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.063299 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:39.150184 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.150153 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" Sep 09 21:01:39.169517 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.169455 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:01:39.170418 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.170403 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" Sep 09 21:01:39.192413 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.192394 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:01:39.529586 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.529498 2578 apiserver.go:52] "Watching apiserver" Sep 09 21:01:39.537903 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.537874 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 21:01:39.538402 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.538365 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-qdmqq","kube-system/global-pull-secret-syncer-vcxpp","openshift-image-registry/node-ca-7lm7d","openshift-multus/network-metrics-daemon-d9g6l","openshift-network-diagnostics/network-check-target-b6dd7","openshift-network-operator/iptables-alerter-lh2lk","openshift-ovn-kubernetes/ovnkube-node-chpbt","kube-system/konnectivity-agent-gvcdp","kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw","openshift-cluster-node-tuning-operator/tuned-mjms8","openshift-dns/node-resolver-rjhvd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal","openshift-multus/multus-additional-cni-plugins-989mt"] Sep 09 21:01:39.541248 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.541226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.543649 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.543623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.544735 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.544716 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 21:01:39.544735 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.544724 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 21:01:39.544892 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.544716 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 21:01:39.545472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.545454 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 21:01:39.545790 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.545762 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wr5v7\"" Sep 09 21:01:39.545957 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.545941 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 21:01:39.546406 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.546388 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 21:01:39.546610 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.546591 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:01:39.546690 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.546671 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8tklf\"" Sep 09 21:01:39.546883 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.546864 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.546981 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.546949 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 21:01:39.548794 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.548589 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:39.548885 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.548862 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:39.549366 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.549350 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 21:01:39.549471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.549451 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 21:01:39.549715 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.549701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mbmcs\"" Sep 09 21:01:39.550470 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.549970 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 21:01:39.550470 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.549996 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 21:01:39.551069 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.551052 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:39.551159 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.551115 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:39.553832 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.553810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.556400 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.556380 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.556644 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.556623 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 21:01:39.556806 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.556792 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:01:39.557090 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.557070 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nkfc7\"" Sep 09 21:01:39.557511 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.557494 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 21:01:39.558648 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.558629 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 21:01:39.558949 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.558930 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-sys\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.559036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.558955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-tuned\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.559036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm88v\" (UniqueName: \"kubernetes.io/projected/85e4a129-3ede-4946-b4a2-2d86b351df38-kube-api-access-pm88v\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.559175 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:39.559175 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-var-lib-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559175 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8a793f9-729c-42cf-9255-24dcbd443b9b-tmp\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.559175 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85e4a129-3ede-4946-b4a2-2d86b351df38-host\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.559175 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/85e4a129-3ede-4946-b4a2-2d86b351df38-serviceca\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559220 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-etc-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-ovn\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-node-log\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559349 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-run-ovn-kubernetes\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559377 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-systemd\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559401 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-cni-bin\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559428 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559434 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-td852\"" Sep 09 21:01:39.559514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559466 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-env-overrides\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559531 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-ovnkube-script-lib\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-systemd\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559606 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-lib-modules\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-kubelet\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-ovnkube-config\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-var-lib-kubelet\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-host\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559788 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8tf\" (UniqueName: \"kubernetes.io/projected/a8a793f9-729c-42cf-9255-24dcbd443b9b-kube-api-access-5p8tf\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559869 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-log-socket\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-systemd-units\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysctl-d\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.559994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysctl-conf\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-run\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.560683 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560123 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 21:01:39.560683 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560427 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 21:01:39.560880 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z246d\" (UniqueName: \"kubernetes.io/projected/a6d9f290-84f2-4160-8260-687ccd27cb13-kube-api-access-z246d\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.560942 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysconfig\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.561002 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6d9f290-84f2-4160-8260-687ccd27cb13-ovn-node-metrics-cert\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.561002 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq7tn\" (UniqueName: \"kubernetes.io/projected/5e6ddbb9-c28f-44da-b932-5f722aab46f4-kube-api-access-dq7tn\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:39.561002 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.560991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-slash\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.561154 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.561013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-cni-netd\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.561154 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.561048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-run-netns\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.561154 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.561079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-modprobe-d\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.561154 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.561115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-kubernetes\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.562597 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.562574 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 21:01:39.563140 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.563124 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.563373 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.563199 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:39.564054 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.564033 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 21:01:39.564365 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.564345 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 21:01:39.564473 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.564431 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 21:01:39.564850 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.564831 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mkp7c\"" Sep 09 21:01:39.565809 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.565784 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:01:39.567754 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.567735 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.570560 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.570538 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 21:01:39.570815 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.570793 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 21:01:39.570994 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.570973 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 21:01:39.571071 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.571056 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.571263 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.571239 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mnj4k\"" Sep 09 21:01:39.573011 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.572990 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.573487 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.573467 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 21:01:39.573810 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.573790 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rrfjq\"" Sep 09 21:01:39.575097 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.575074 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 21:01:39.575574 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.575556 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 21:01:39.576688 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.576669 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jnlkr\"" Sep 09 21:01:39.578377 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.578355 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:39.593536 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.593512 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jk8xs" Sep 09 21:01:39.602614 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.602594 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jk8xs" Sep 09 21:01:39.651233 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.651206 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 21:01:39.661332 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.661430 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/109f2c1a-0905-49d5-bb38-092b9c9130a5-agent-certs\") pod \"konnectivity-agent-gvcdp\" (UID: \"109f2c1a-0905-49d5-bb38-092b9c9130a5\") " pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.661430 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-var-lib-kubelet\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.661430 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8tf\" (UniqueName: \"kubernetes.io/projected/a8a793f9-729c-42cf-9255-24dcbd443b9b-kube-api-access-5p8tf\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.661577 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661441 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-kubelet-config\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.661577 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfjs2\" (UniqueName: \"kubernetes.io/projected/00dbb388-31d8-4461-ba75-396770d7720e-kube-api-access-mfjs2\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.661577 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-systemd-units\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.661577 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661500 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-var-lib-kubelet\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.661577 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-run\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.661577 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z246d\" (UniqueName: \"kubernetes.io/projected/a6d9f290-84f2-4160-8260-687ccd27cb13-kube-api-access-z246d\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661589 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-systemd-units\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-socket-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-system-cni-dir\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/109f2c1a-0905-49d5-bb38-092b9c9130a5-konnectivity-ca\") pod \"konnectivity-agent-gvcdp\" (UID: \"109f2c1a-0905-49d5-bb38-092b9c9130a5\") " pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661740 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-run\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/010ef23c-e565-4385-89a7-6c29b18be4ff-host-slash\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq7tn\" (UniqueName: \"kubernetes.io/projected/5e6ddbb9-c28f-44da-b932-5f722aab46f4-kube-api-access-dq7tn\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-slash\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-cni-netd\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-dbus\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661903 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-slash\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.661919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-cnibin\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661938 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-cni-netd\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-cni-multus\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.661985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-run-netns\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662010 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-modprobe-d\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-sys\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-tuned\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662058 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-run-netns\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662090 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-run-ovn-kubernetes\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662106 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-sys\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/010ef23c-e565-4385-89a7-6c29b18be4ff-iptables-alerter-script\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662144 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctc4v\" (UniqueName: \"kubernetes.io/projected/010ef23c-e565-4385-89a7-6c29b18be4ff-kube-api-access-ctc4v\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662150 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-run-ovn-kubernetes\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662156 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-modprobe-d\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662177 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-env-overrides\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8a793f9-729c-42cf-9255-24dcbd443b9b-tmp\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.662609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85e4a129-3ede-4946-b4a2-2d86b351df38-host\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662437 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85e4a129-3ede-4946-b4a2-2d86b351df38-host\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662468 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/85e4a129-3ede-4946-b4a2-2d86b351df38-serviceca\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-node-log\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662534 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-k8s-cni-cncf-io\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-ovnkube-script-lib\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-multus-certs\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-hosts-file\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662654 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-node-log\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-tmp-dir\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662710 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-env-overrides\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-lib-modules\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662749 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-kubelet\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-ovnkube-config\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-cnibin\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662836 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-kubelet\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.663471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-host\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662896 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/85e4a129-3ede-4946-b4a2-2d86b351df38-serviceca\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662922 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-lib-modules\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-log-socket\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-host\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-socket-dir-parent\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-log-socket\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.662994 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-cni-bin\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-kubelet\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysctl-d\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysctl-conf\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-device-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysconfig\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6d9f290-84f2-4160-8260-687ccd27cb13-ovn-node-metrics-cert\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysctl-d\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.664433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663200 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6931fd17-6b9c-4432-8bc2-8d97a9a11784-cni-binary-copy\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663202 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysconfig\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-hostroot\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663280 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-sysctl-conf\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-daemon-config\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663323 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-ovnkube-script-lib\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-cni-binary-copy\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663372 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-registration-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd2p9\" (UniqueName: \"kubernetes.io/projected/737b1a9a-fc77-47da-9a9f-147220a033ee-kube-api-access-vd2p9\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-kubernetes\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663443 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a6d9f290-84f2-4160-8260-687ccd27cb13-ovnkube-config\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm88v\" (UniqueName: \"kubernetes.io/projected/85e4a129-3ede-4946-b4a2-2d86b351df38-kube-api-access-pm88v\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-var-lib-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-etc-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-cni-bin\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665037 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663581 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-kubernetes\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.663605 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.663668 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:40.163647998 +0000 UTC m=+2.977498151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-ovn\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663786 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-netns\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663829 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-etc-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663830 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-systemd\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-sys-fs\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663889 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-cni-bin\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663894 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-etc-kubernetes\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-ovn\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663940 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ddgz\" (UniqueName: \"kubernetes.io/projected/6931fd17-6b9c-4432-8bc2-8d97a9a11784-kube-api-access-4ddgz\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663959 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-var-lib-openvswitch\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663870 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a6d9f290-84f2-4160-8260-687ccd27cb13-run-systemd\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.665716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.663982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l77mt\" (UniqueName: \"kubernetes.io/projected/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-kube-api-access-l77mt\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.666205 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.664012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-systemd\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.666205 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.664049 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-systemd\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.666205 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.664076 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-system-cni-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.666205 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.664101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-cni-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.666205 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.664117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-conf-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.667256 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.667227 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a6d9f290-84f2-4160-8260-687ccd27cb13-ovn-node-metrics-cert\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.667256 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.667237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8a793f9-729c-42cf-9255-24dcbd443b9b-etc-tuned\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.668434 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.668412 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8a793f9-729c-42cf-9255-24dcbd443b9b-tmp\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.672491 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.672437 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq7tn\" (UniqueName: \"kubernetes.io/projected/5e6ddbb9-c28f-44da-b932-5f722aab46f4-kube-api-access-dq7tn\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:39.672607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.672522 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8tf\" (UniqueName: \"kubernetes.io/projected/a8a793f9-729c-42cf-9255-24dcbd443b9b-kube-api-access-5p8tf\") pod \"tuned-mjms8\" (UID: \"a8a793f9-729c-42cf-9255-24dcbd443b9b\") " pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.672607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.672550 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z246d\" (UniqueName: \"kubernetes.io/projected/a6d9f290-84f2-4160-8260-687ccd27cb13-kube-api-access-z246d\") pod \"ovnkube-node-chpbt\" (UID: \"a6d9f290-84f2-4160-8260-687ccd27cb13\") " pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.674629 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.674550 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:39.674726 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.674634 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:39.674726 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.674644 2578 projected.go:196] Error preparing data for projected volume kube-api-access-w6k97 for pod openshift-network-diagnostics/network-check-target-b6dd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:39.674726 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.674708 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97 podName:f018ab5a-68d5-42d0-9f15-2874b76a0ae1 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:40.174679973 +0000 UTC m=+2.988530120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w6k97" (UniqueName: "kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97") pod "network-check-target-b6dd7" (UID: "f018ab5a-68d5-42d0-9f15-2874b76a0ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:39.677292 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.677273 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm88v\" (UniqueName: \"kubernetes.io/projected/85e4a129-3ede-4946-b4a2-2d86b351df38-kube-api-access-pm88v\") pod \"node-ca-7lm7d\" (UID: \"85e4a129-3ede-4946-b4a2-2d86b351df38\") " pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.686330 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.686265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" event={"ID":"548898899e6221ee528d5a8d7f51c774","Type":"ContainerStarted","Data":"a4516f7191cc513b7d5540581a7efb2f009cd973ad7f957fa82605e04ab3d531"} Sep 09 21:01:39.687770 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.687745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" event={"ID":"0559b432e0a8e8090650f241aa697b78","Type":"ContainerStarted","Data":"6f4b1c14a4b919945745772743df2afc720ee58fd5e87eee1a02b1e87db2b611"} Sep 09 21:01:39.764470 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-registration-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.764470 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764471 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd2p9\" (UniqueName: \"kubernetes.io/projected/737b1a9a-fc77-47da-9a9f-147220a033ee-kube-api-access-vd2p9\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.764641 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-netns\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764641 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764553 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-netns\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764641 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764566 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-sys-fs\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.764641 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-etc-kubernetes\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764641 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764622 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-sys-fs\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.764641 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ddgz\" (UniqueName: \"kubernetes.io/projected/6931fd17-6b9c-4432-8bc2-8d97a9a11784-kube-api-access-4ddgz\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764653 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-etc-kubernetes\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764572 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-registration-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764667 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l77mt\" (UniqueName: \"kubernetes.io/projected/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-kube-api-access-l77mt\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-system-cni-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764716 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-cni-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764738 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-conf-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/109f2c1a-0905-49d5-bb38-092b9c9130a5-agent-certs\") pod \"konnectivity-agent-gvcdp\" (UID: \"109f2c1a-0905-49d5-bb38-092b9c9130a5\") " pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-system-cni-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764807 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-conf-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764818 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-kubelet-config\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-cni-dir\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764886 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-kubelet-config\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfjs2\" (UniqueName: \"kubernetes.io/projected/00dbb388-31d8-4461-ba75-396770d7720e-kube-api-access-mfjs2\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.764982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-socket-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.764986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765001 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-system-cni-dir\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765032 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/109f2c1a-0905-49d5-bb38-092b9c9130a5-konnectivity-ca\") pod \"konnectivity-agent-gvcdp\" (UID: \"109f2c1a-0905-49d5-bb38-092b9c9130a5\") " pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/010ef23c-e565-4385-89a7-6c29b18be4ff-host-slash\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-socket-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-dbus\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-cnibin\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765138 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-system-cni-dir\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765141 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/010ef23c-e565-4385-89a7-6c29b18be4ff-host-slash\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-cni-multus\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765193 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-cnibin\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-dbus\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/010ef23c-e565-4385-89a7-6c29b18be4ff-iptables-alerter-script\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctc4v\" (UniqueName: \"kubernetes.io/projected/010ef23c-e565-4385-89a7-6c29b18be4ff-kube-api-access-ctc4v\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765266 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-cni-multus\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.765827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-k8s-cni-cncf-io\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-multus-certs\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-hosts-file\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-tmp-dir\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-cnibin\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765437 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-k8s-cni-cncf-io\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-socket-dir-parent\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765506 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-run-multus-certs\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-cni-bin\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765558 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-hosts-file\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765571 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-socket-dir-parent\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-kubelet\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765635 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-device-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765666 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-kubelet\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765717 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-device-dir\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-host-var-lib-cni-bin\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.766605 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.765737 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765761 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-tmp-dir\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:39.765792 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret podName:6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:40.265774659 +0000 UTC m=+3.079624787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret") pod "global-pull-secret-syncer-vcxpp" (UID: "6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765667 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765822 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-cnibin\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765785 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/109f2c1a-0905-49d5-bb38-092b9c9130a5-konnectivity-ca\") pod \"konnectivity-agent-gvcdp\" (UID: \"109f2c1a-0905-49d5-bb38-092b9c9130a5\") " pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/737b1a9a-fc77-47da-9a9f-147220a033ee-etc-selinux\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765825 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6931fd17-6b9c-4432-8bc2-8d97a9a11784-cni-binary-copy\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-hostroot\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765901 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00dbb388-31d8-4461-ba75-396770d7720e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-daemon-config\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765911 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/010ef23c-e565-4385-89a7-6c29b18be4ff-iptables-alerter-script\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6931fd17-6b9c-4432-8bc2-8d97a9a11784-hostroot\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.765995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-cni-binary-copy\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.766321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6931fd17-6b9c-4432-8bc2-8d97a9a11784-cni-binary-copy\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.766580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.766669 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00dbb388-31d8-4461-ba75-396770d7720e-cni-binary-copy\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.767336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.767009 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6931fd17-6b9c-4432-8bc2-8d97a9a11784-multus-daemon-config\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.768057 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.767772 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/109f2c1a-0905-49d5-bb38-092b9c9130a5-agent-certs\") pod \"konnectivity-agent-gvcdp\" (UID: \"109f2c1a-0905-49d5-bb38-092b9c9130a5\") " pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.780593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.780531 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctc4v\" (UniqueName: \"kubernetes.io/projected/010ef23c-e565-4385-89a7-6c29b18be4ff-kube-api-access-ctc4v\") pod \"iptables-alerter-lh2lk\" (UID: \"010ef23c-e565-4385-89a7-6c29b18be4ff\") " pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.781898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.781848 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l77mt\" (UniqueName: \"kubernetes.io/projected/7f2d556d-c2dc-4e5f-b243-b320d11c24d4-kube-api-access-l77mt\") pod \"node-resolver-rjhvd\" (UID: \"7f2d556d-c2dc-4e5f-b243-b320d11c24d4\") " pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:39.781898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.781874 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd2p9\" (UniqueName: \"kubernetes.io/projected/737b1a9a-fc77-47da-9a9f-147220a033ee-kube-api-access-vd2p9\") pod \"aws-ebs-csi-driver-node-h2gxw\" (UID: \"737b1a9a-fc77-47da-9a9f-147220a033ee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.782039 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.781934 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfjs2\" (UniqueName: \"kubernetes.io/projected/00dbb388-31d8-4461-ba75-396770d7720e-kube-api-access-mfjs2\") pod \"multus-additional-cni-plugins-989mt\" (UID: \"00dbb388-31d8-4461-ba75-396770d7720e\") " pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.782039 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.781955 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ddgz\" (UniqueName: \"kubernetes.io/projected/6931fd17-6b9c-4432-8bc2-8d97a9a11784-kube-api-access-4ddgz\") pod \"multus-qdmqq\" (UID: \"6931fd17-6b9c-4432-8bc2-8d97a9a11784\") " pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.854982 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.854952 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:01:39.863396 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.863369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mjms8" Sep 09 21:01:39.874221 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.874177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7lm7d" Sep 09 21:01:39.881190 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.881173 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lh2lk" Sep 09 21:01:39.889970 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.889950 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-989mt" Sep 09 21:01:39.897739 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.897722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:01:39.905549 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.905527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" Sep 09 21:01:39.913518 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.913481 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qdmqq" Sep 09 21:01:39.920360 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:39.920341 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rjhvd" Sep 09 21:01:40.169266 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.169164 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:40.169419 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.169325 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:40.169419 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.169409 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:41.169387447 +0000 UTC m=+3.983237602 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:40.269687 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.269652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:40.269687 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.269704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:40.269927 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.269832 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:40.269927 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.269844 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:40.269927 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.269869 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:40.269927 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.269884 2578 projected.go:196] Error preparing data for projected volume kube-api-access-w6k97 for pod openshift-network-diagnostics/network-check-target-b6dd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:40.269927 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.269918 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret podName:6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:41.269892764 +0000 UTC m=+4.083742893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret") pod "global-pull-secret-syncer-vcxpp" (UID: "6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:40.270149 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:40.269940 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97 podName:f018ab5a-68d5-42d0-9f15-2874b76a0ae1 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:41.269926495 +0000 UTC m=+4.083776649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w6k97" (UniqueName: "kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97") pod "network-check-target-b6dd7" (UID: "f018ab5a-68d5-42d0-9f15-2874b76a0ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:40.480561 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:40.480532 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010ef23c_e565_4385_89a7_6c29b18be4ff.slice/crio-bca5631de795e576c0bfce8d698752b9f3a04a273a56626918bca5fb09ad8f1b WatchSource:0}: Error finding container bca5631de795e576c0bfce8d698752b9f3a04a273a56626918bca5fb09ad8f1b: Status 404 returned error can't find the container with id bca5631de795e576c0bfce8d698752b9f3a04a273a56626918bca5fb09ad8f1b Sep 09 21:01:40.493893 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:40.493853 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737b1a9a_fc77_47da_9a9f_147220a033ee.slice/crio-6a42e6a32e9dfc87bb9db243a0631bbf6ce3f80147adbc9779828bf10378f884 WatchSource:0}: Error finding container 6a42e6a32e9dfc87bb9db243a0631bbf6ce3f80147adbc9779828bf10378f884: Status 404 returned error can't find the container with id 6a42e6a32e9dfc87bb9db243a0631bbf6ce3f80147adbc9779828bf10378f884 Sep 09 21:01:40.516246 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:40.516212 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e4a129_3ede_4946_b4a2_2d86b351df38.slice/crio-c0623f724fd7a33abe1f096a0407e6a910582cfff4338645b676f804032c41c0 WatchSource:0}: Error finding container c0623f724fd7a33abe1f096a0407e6a910582cfff4338645b676f804032c41c0: Status 404 returned error can't find the container with id c0623f724fd7a33abe1f096a0407e6a910582cfff4338645b676f804032c41c0 Sep 09 21:01:40.517744 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:40.517418 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2d556d_c2dc_4e5f_b243_b320d11c24d4.slice/crio-ad8785faaeccb9ebef3dda5adb90052e10a29da10426add1fe3c0a3f51de289a WatchSource:0}: Error finding container ad8785faaeccb9ebef3dda5adb90052e10a29da10426add1fe3c0a3f51de289a: Status 404 returned error can't find the container with id ad8785faaeccb9ebef3dda5adb90052e10a29da10426add1fe3c0a3f51de289a Sep 09 21:01:40.518828 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:40.518554 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109f2c1a_0905_49d5_bb38_092b9c9130a5.slice/crio-8f40a7c9bb2fd847998ded1346dd7ae0740c7b799e6d21316bea17c60c2e8b0b WatchSource:0}: Error finding container 8f40a7c9bb2fd847998ded1346dd7ae0740c7b799e6d21316bea17c60c2e8b0b: Status 404 returned error can't find the container with id 8f40a7c9bb2fd847998ded1346dd7ae0740c7b799e6d21316bea17c60c2e8b0b Sep 09 21:01:40.519446 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:01:40.519419 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6931fd17_6b9c_4432_8bc2_8d97a9a11784.slice/crio-4a90376e45721227cd69372c8b88d4950a6d686f3761fffa5e55c999cbfea7fe WatchSource:0}: Error finding container 4a90376e45721227cd69372c8b88d4950a6d686f3761fffa5e55c999cbfea7fe: Status 404 returned error can't find the container with id 4a90376e45721227cd69372c8b88d4950a6d686f3761fffa5e55c999cbfea7fe Sep 09 21:01:40.606523 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.606487 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:56:40 +0000 UTC" deadline="2028-03-09 18:54:54.43055565 +0000 UTC" Sep 09 21:01:40.606523 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.606523 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13125h53m13.824037171s" Sep 09 21:01:40.691197 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.691161 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qdmqq" event={"ID":"6931fd17-6b9c-4432-8bc2-8d97a9a11784","Type":"ContainerStarted","Data":"4a90376e45721227cd69372c8b88d4950a6d686f3761fffa5e55c999cbfea7fe"} Sep 09 21:01:40.692644 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.692613 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gvcdp" event={"ID":"109f2c1a-0905-49d5-bb38-092b9c9130a5","Type":"ContainerStarted","Data":"8f40a7c9bb2fd847998ded1346dd7ae0740c7b799e6d21316bea17c60c2e8b0b"} Sep 09 21:01:40.694177 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.694147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7lm7d" event={"ID":"85e4a129-3ede-4946-b4a2-2d86b351df38","Type":"ContainerStarted","Data":"c0623f724fd7a33abe1f096a0407e6a910582cfff4338645b676f804032c41c0"} Sep 09 21:01:40.695597 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.695557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" event={"ID":"737b1a9a-fc77-47da-9a9f-147220a033ee","Type":"ContainerStarted","Data":"6a42e6a32e9dfc87bb9db243a0631bbf6ce3f80147adbc9779828bf10378f884"} Sep 09 21:01:40.697111 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.697088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"7c1cb66c83e9657e4a3b35d0dfb565a80c85217ecaf06216eb4a553d09f07dcb"} Sep 09 21:01:40.699888 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.699825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" event={"ID":"0559b432e0a8e8090650f241aa697b78","Type":"ContainerStarted","Data":"28a554b0501a42821cd6b568d5894911bb634a240805447f6fec89750fc24d9b"} Sep 09 21:01:40.701382 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.701358 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjhvd" event={"ID":"7f2d556d-c2dc-4e5f-b243-b320d11c24d4","Type":"ContainerStarted","Data":"ad8785faaeccb9ebef3dda5adb90052e10a29da10426add1fe3c0a3f51de289a"} Sep 09 21:01:40.705682 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.702918 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerStarted","Data":"73bacf256da8273b621686bed3de0bd3fae0dd4dba2073750c4de790fbcc0c6b"} Sep 09 21:01:40.707505 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.707475 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mjms8" event={"ID":"a8a793f9-729c-42cf-9255-24dcbd443b9b","Type":"ContainerStarted","Data":"3ff6210b14b87ccae55b929b2bd0ee9158d923b1a33cffd6fabdc7c475e12f67"} Sep 09 21:01:40.708711 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:40.708683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lh2lk" event={"ID":"010ef23c-e565-4385-89a7-6c29b18be4ff","Type":"ContainerStarted","Data":"bca5631de795e576c0bfce8d698752b9f3a04a273a56626918bca5fb09ad8f1b"} Sep 09 21:01:41.177598 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.177556 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:41.177803 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.177740 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:41.177874 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.177808 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:43.177787889 +0000 UTC m=+5.991638033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.278692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.278777 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.278903 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.278968 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret podName:6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:43.278948626 +0000 UTC m=+6.092798771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret") pod "global-pull-secret-syncer-vcxpp" (UID: "6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.279421 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.279441 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.279455 2578 projected.go:196] Error preparing data for projected volume kube-api-access-w6k97 for pod openshift-network-diagnostics/network-check-target-b6dd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:41.279760 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.279501 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97 podName:f018ab5a-68d5-42d0-9f15-2874b76a0ae1 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:43.279485121 +0000 UTC m=+6.093335265 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w6k97" (UniqueName: "kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97") pod "network-check-target-b6dd7" (UID: "f018ab5a-68d5-42d0-9f15-2874b76a0ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:41.323482 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.323255 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:01:41.608439 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.607231 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:56:40 +0000 UTC" deadline="2028-05-29 12:23:53.049133418 +0000 UTC" Sep 09 21:01:41.608439 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.607294 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15063h22m11.441843679s" Sep 09 21:01:41.684801 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.682459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:41.684801 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.682596 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:41.684801 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.683023 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:41.684801 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.683134 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:41.684801 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:41.683343 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:41.684801 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:41.683414 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:43.197600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:43.197560 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:43.198089 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.197769 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:43.198089 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.197836 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:47.197815532 +0000 UTC m=+10.011665670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:43.299055 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:43.299010 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:43.299534 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:43.299080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:43.299534 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.299248 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:43.299534 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.299268 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:43.299534 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.299281 2578 projected.go:196] Error preparing data for projected volume kube-api-access-w6k97 for pod openshift-network-diagnostics/network-check-target-b6dd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:43.299534 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.299344 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97 podName:f018ab5a-68d5-42d0-9f15-2874b76a0ae1 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:47.299324366 +0000 UTC m=+10.113174527 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w6k97" (UniqueName: "kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97") pod "network-check-target-b6dd7" (UID: "f018ab5a-68d5-42d0-9f15-2874b76a0ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:43.299534 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.299346 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:43.299534 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.299417 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret podName:6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:47.299397677 +0000 UTC m=+10.113247831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret") pod "global-pull-secret-syncer-vcxpp" (UID: "6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:43.683467 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:43.683384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:43.683631 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.683540 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:43.683971 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:43.683951 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:43.684080 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.684058 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:43.684149 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:43.684120 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:43.684205 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:43.684188 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:45.685661 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:45.684852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:45.685661 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:45.684995 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:45.685661 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:45.685404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:45.685661 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:45.685497 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:45.685661 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:45.685544 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:45.685661 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:45.685611 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:47.234410 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:47.234368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:47.234903 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.234553 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:47.234903 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.234630 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:55.234607023 +0000 UTC m=+18.048457158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:47.335612 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:47.335522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:47.335798 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:47.335725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:47.336500 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.335908 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:47.336500 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.335930 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:47.336500 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.335944 2578 projected.go:196] Error preparing data for projected volume kube-api-access-w6k97 for pod openshift-network-diagnostics/network-check-target-b6dd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:47.336500 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.335992 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97 podName:f018ab5a-68d5-42d0-9f15-2874b76a0ae1 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:55.335979214 +0000 UTC m=+18.149829356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w6k97" (UniqueName: "kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97") pod "network-check-target-b6dd7" (UID: "f018ab5a-68d5-42d0-9f15-2874b76a0ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:47.336500 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.336416 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:47.336500 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.336469 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret podName:6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5 nodeName:}" failed. No retries permitted until 2026-09-09 21:01:55.336453123 +0000 UTC m=+18.150303254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret") pod "global-pull-secret-syncer-vcxpp" (UID: "6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:47.682251 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:47.682149 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:47.682431 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.682271 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:47.682667 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:47.682646 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:47.682782 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.682760 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:47.682879 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:47.682865 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:47.682965 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:47.682947 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:49.684413 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:49.684382 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:49.684978 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:49.684443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:49.684978 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:49.684557 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:49.684978 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:49.684568 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:49.684978 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:49.684645 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:49.684978 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:49.684707 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:51.681559 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:51.681523 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:51.681559 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:51.681527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:51.682057 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:51.681537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:51.682057 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:51.681645 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:51.682057 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:51.681774 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:51.682057 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:51.681847 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:53.681951 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:53.681918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:53.681951 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:53.681949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:53.682448 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:53.681925 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:53.682448 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:53.682041 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:53.682448 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:53.682092 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:53.682448 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:53.682214 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:55.292683 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:55.292648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:55.293138 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.292819 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:55.293138 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.292890 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:11.292874695 +0000 UTC m=+34.106724822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:01:55.393659 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:55.393614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:55.393819 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:55.393694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:55.393819 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.393811 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:55.393915 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.393814 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:01:55.393915 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.393840 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:01:55.393915 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.393853 2578 projected.go:196] Error preparing data for projected volume kube-api-access-w6k97 for pod openshift-network-diagnostics/network-check-target-b6dd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:55.393915 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.393867 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret podName:6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:11.393853295 +0000 UTC m=+34.207703436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret") pod "global-pull-secret-syncer-vcxpp" (UID: "6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:01:55.393915 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.393904 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97 podName:f018ab5a-68d5-42d0-9f15-2874b76a0ae1 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:11.393888389 +0000 UTC m=+34.207738519 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w6k97" (UniqueName: "kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97") pod "network-check-target-b6dd7" (UID: "f018ab5a-68d5-42d0-9f15-2874b76a0ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:01:55.682389 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:55.682274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:55.682389 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:55.682289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:55.682593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:55.682426 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:55.682593 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.682426 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:55.682593 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.682537 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:55.682750 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:55.682611 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:57.682017 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:57.681980 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:57.682495 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:57.682088 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:57.682495 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:57.682178 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:57.682495 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:57.682293 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:57.682495 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:57.682340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:57.682495 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:57.682419 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:59.682769 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.682223 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:01:59.682769 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:59.682549 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:01:59.682769 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.682363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:01:59.682769 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:59.682649 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:01:59.682769 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.682233 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:01:59.682769 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:01:59.682736 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:01:59.762919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.762892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"5476cae2eb6f26bf28e0c18bce386c43743767504ddd0012a588c33a1c5d70ee"} Sep 09 21:01:59.763036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.762963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"c77767ca0882445f42a88df8216108ac927bffad8f9213337e88d2f589803d51"} Sep 09 21:01:59.763036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.763001 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"b7afb31543db51116310f49d6d58709bb4dc4c8b3b7dd6577915392d61b5de53"} Sep 09 21:01:59.764224 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.764184 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mjms8" event={"ID":"a8a793f9-729c-42cf-9255-24dcbd443b9b","Type":"ContainerStarted","Data":"64489876be48fece23f7416889542ca918d2fd8840ad4691010b47596202b39f"} Sep 09 21:01:59.765521 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.765503 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qdmqq" event={"ID":"6931fd17-6b9c-4432-8bc2-8d97a9a11784","Type":"ContainerStarted","Data":"039bf2b0dcd1bb62c7b9bd721c16b918a3d928a40924c3b9590351c48676bdb9"} Sep 09 21:01:59.803213 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.803136 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-18.ec2.internal" podStartSLOduration=19.803116034 podStartE2EDuration="19.803116034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:40.719293234 +0000 UTC m=+3.533143387" watchObservedRunningTime="2026-09-09 21:01:59.803116034 +0000 UTC m=+22.616966185" Sep 09 21:01:59.854615 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.854555 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mjms8" podStartSLOduration=2.942598338 podStartE2EDuration="21.854534351s" totalImagesPullingTime="18.911936013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:59.8067473 +0000 UTC m=+22.620597451" watchObservedRunningTime="2026-09-09 21:01:59.854534351 +0000 UTC m=+22.668384501" Sep 09 21:01:59.854940 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:01:59.854900 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qdmqq" podStartSLOduration=2.940534547 podStartE2EDuration="21.854886985s" totalImagesPullingTime="18.914352438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:59.853535014 +0000 UTC m=+22.667385174" watchObservedRunningTime="2026-09-09 21:01:59.854886985 +0000 UTC m=+22.668737135" Sep 09 21:02:00.768870 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.768841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rjhvd" event={"ID":"7f2d556d-c2dc-4e5f-b243-b320d11c24d4","Type":"ContainerStarted","Data":"5948440f3e695674fca120cb8c6b1674d57e27ee704311e3a9c213c8f2c3f69c"} Sep 09 21:02:00.770372 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.770334 2578 generic.go:356] "Generic (PLEG): container finished" podID="00dbb388-31d8-4461-ba75-396770d7720e" containerID="86a0e954f22a4417c796e240a357b1b679d2d0eaf600ed1dc79e23f340b6931b" exitCode=0 Sep 09 21:02:00.770460 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.770386 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerDied","Data":"86a0e954f22a4417c796e240a357b1b679d2d0eaf600ed1dc79e23f340b6931b"} Sep 09 21:02:00.773607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.773587 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gvcdp" event={"ID":"109f2c1a-0905-49d5-bb38-092b9c9130a5","Type":"ContainerStarted","Data":"609cb63670cd4f55c47604d69a2948e9dbd53929f85410825065e3d639d98035"} Sep 09 21:02:00.774946 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.774926 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7lm7d" event={"ID":"85e4a129-3ede-4946-b4a2-2d86b351df38","Type":"ContainerStarted","Data":"523579f1a44e72db2589d521dfd447127e034b581b409c59829c0a6d3cf87104"} Sep 09 21:02:00.776384 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.776364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" event={"ID":"737b1a9a-fc77-47da-9a9f-147220a033ee","Type":"ContainerStarted","Data":"ac4a74712034224f41d446d76da47a8f127bc39d08f065a834fd84bbaba13526"} Sep 09 21:02:00.779361 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.779336 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"5ab6711ad70e8ac2316e25081b63f4dd99c8dae95040c272ca8f533c7cb782ce"} Sep 09 21:02:00.779445 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.779396 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"7c17464303859df0d2d05e0b358fb2bb2dfaea04f2c715a1e4e7c468d2d87c60"} Sep 09 21:02:00.779499 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.779441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"a206c7f76aad3c1e453b50216ed30edb86fc70552faf35cc459e1a41c9fa6540"} Sep 09 21:02:00.780616 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.780594 2578 generic.go:356] "Generic (PLEG): container finished" podID="548898899e6221ee528d5a8d7f51c774" containerID="6496bb665af44b5557fe296992573eb7d617b928a5dd864289461c967c09674e" exitCode=0 Sep 09 21:02:00.780694 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.780665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" event={"ID":"548898899e6221ee528d5a8d7f51c774","Type":"ContainerDied","Data":"6496bb665af44b5557fe296992573eb7d617b928a5dd864289461c967c09674e"} Sep 09 21:02:00.788778 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.788741 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rjhvd" podStartSLOduration=3.922321395 podStartE2EDuration="22.788723769s" totalImagesPullingTime="18.866402374s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:00.788597396 +0000 UTC m=+23.602447545" watchObservedRunningTime="2026-09-09 21:02:00.788723769 +0000 UTC m=+23.602573918" Sep 09 21:02:00.808904 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.808873 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gvcdp" podStartSLOduration=11.037917746 podStartE2EDuration="22.808860737s" totalImagesPullingTime="11.770942991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:00.808336677 +0000 UTC m=+23.622186826" watchObservedRunningTime="2026-09-09 21:02:00.808860737 +0000 UTC m=+23.622710886" Sep 09 21:02:00.844593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:00.844553 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7lm7d" podStartSLOduration=3.97312618 podStartE2EDuration="22.844540523s" totalImagesPullingTime="18.871414343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:00.844014575 +0000 UTC m=+23.657864725" watchObservedRunningTime="2026-09-09 21:02:00.844540523 +0000 UTC m=+23.658390671" Sep 09 21:02:01.682706 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:01.682244 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:01.682706 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:01.682286 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:01.683129 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:01.682362 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:02:01.683129 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:01.682245 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:01.683129 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:01.682453 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:02:01.683129 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:01.683100 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:02:01.785101 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:01.784902 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" event={"ID":"548898899e6221ee528d5a8d7f51c774","Type":"ContainerStarted","Data":"06d0b3d727dab13e74cdfbdfae6bf7b111c94f111305cd94ec67cad9c03349c7"} Sep 09 21:02:01.787506 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:01.787471 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lh2lk" event={"ID":"010ef23c-e565-4385-89a7-6c29b18be4ff","Type":"ContainerStarted","Data":"8887d133e68c48193f9dad5c5c47835dde7f6188c4c266e85a72032e0e126bf6"} Sep 09 21:02:01.836421 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:01.836369 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-18.ec2.internal" podStartSLOduration=21.83635336 podStartE2EDuration="21.83635336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:01.810760498 +0000 UTC m=+24.624610646" watchObservedRunningTime="2026-09-09 21:02:01.83635336 +0000 UTC m=+24.650203487" Sep 09 21:02:02.266293 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.266257 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:02:02.267039 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.267013 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:02:02.295128 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.295080 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lh2lk" podStartSLOduration=5.417020506 podStartE2EDuration="24.295065702s" totalImagesPullingTime="18.878045196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:01.837014128 +0000 UTC m=+24.650864278" watchObservedRunningTime="2026-09-09 21:02:02.295065702 +0000 UTC m=+25.108915852" Sep 09 21:02:02.424890 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.424612 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 21:02:02.628372 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.628249 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T21:02:02.424633221Z","UUID":"39bf9280-4bc2-42f1-9ea9-cf7153183e49","Handler":null,"Name":"","Endpoint":""} Sep 09 21:02:02.629938 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.629917 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 21:02:02.630069 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.629994 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 21:02:02.791940 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.791906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" event={"ID":"737b1a9a-fc77-47da-9a9f-147220a033ee","Type":"ContainerStarted","Data":"174c480d738c16b0162f9e55bb5f8112c6ff0942197e256724feac0b25995b48"} Sep 09 21:02:02.795981 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.795943 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"7f04f2d110f5965645c2973995ef51d38cc8475e9379b63f4e07c7302acd5726"} Sep 09 21:02:02.796529 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.796502 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:02:02.796639 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:02.796569 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gvcdp" Sep 09 21:02:03.681630 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:03.681590 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:03.681630 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:03.681634 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:03.681847 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:03.681589 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:03.681847 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:03.681731 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:02:03.681847 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:03.681821 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:02:03.681959 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:03.681923 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:02:05.681849 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.681657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:05.682294 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.681728 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:05.682294 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:05.681931 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:02:05.682294 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.681737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:05.682294 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:05.682021 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:02:05.682294 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:05.682081 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:02:05.807915 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.807890 2578 generic.go:356] "Generic (PLEG): container finished" podID="00dbb388-31d8-4461-ba75-396770d7720e" containerID="e982a2988b6ddea82dcdeccdd2b7ea96a6846031f5a9c90945d5d7f75b641fed" exitCode=0 Sep 09 21:02:05.808023 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.807959 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerDied","Data":"e982a2988b6ddea82dcdeccdd2b7ea96a6846031f5a9c90945d5d7f75b641fed"} Sep 09 21:02:05.810262 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.810230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" event={"ID":"737b1a9a-fc77-47da-9a9f-147220a033ee","Type":"ContainerStarted","Data":"463f96cfa8ae5d3390bba1626f38a67a76de897b7f37b2a05606c96c3c4aa384"} Sep 09 21:02:05.813829 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.813810 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" event={"ID":"a6d9f290-84f2-4160-8260-687ccd27cb13","Type":"ContainerStarted","Data":"d1b868a6a9eba4fc6690f66a01ca462030110479276e56f5a32242ffb2a3676f"} Sep 09 21:02:05.814089 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.814069 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:02:05.814159 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.814131 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:02:05.814206 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.814160 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:02:05.830033 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.830015 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:02:05.830119 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.830110 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:02:05.866042 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:05.865994 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h2gxw" podStartSLOduration=3.32077353 podStartE2EDuration="27.865980756s" totalImagesPullingTime="24.545207226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:05.865509053 +0000 UTC m=+28.679359203" watchObservedRunningTime="2026-09-09 21:02:05.865980756 +0000 UTC m=+28.679830905" Sep 09 21:02:07.076005 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.075789 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" podStartSLOduration=10.141088648 podStartE2EDuration="29.075771482s" totalImagesPullingTime="18.934682834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:05.898997476 +0000 UTC m=+28.712847626" watchObservedRunningTime="2026-09-09 21:02:07.075771482 +0000 UTC m=+29.889621631" Sep 09 21:02:07.076729 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.076557 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcxpp"] Sep 09 21:02:07.076729 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.076687 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:07.076837 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:07.076791 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:02:07.077089 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.077063 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6dd7"] Sep 09 21:02:07.077223 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.077152 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:07.077314 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:07.077261 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:02:07.101463 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.101368 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d9g6l"] Sep 09 21:02:07.101560 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.101478 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:07.101597 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:07.101570 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:02:07.820355 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.820325 2578 generic.go:356] "Generic (PLEG): container finished" podID="00dbb388-31d8-4461-ba75-396770d7720e" containerID="49da68f07d7914b5c6cb2a04c2dfe55579cc00111cb1ec1b5f6e4940a3b71eda" exitCode=0 Sep 09 21:02:07.820512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:07.820416 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerDied","Data":"49da68f07d7914b5c6cb2a04c2dfe55579cc00111cb1ec1b5f6e4940a3b71eda"} Sep 09 21:02:08.681733 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:08.681701 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:08.681733 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:08.681736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:08.682228 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:08.681736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:08.682228 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:08.681805 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:02:08.682228 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:08.681889 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:02:08.682228 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:08.681937 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:02:10.681779 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:10.681717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:10.682142 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:10.681717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:10.682142 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:10.681816 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:02:10.682142 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:10.681882 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:02:10.682142 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:10.681717 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:10.682142 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:10.681981 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:02:10.828871 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:10.828844 2578 generic.go:356] "Generic (PLEG): container finished" podID="00dbb388-31d8-4461-ba75-396770d7720e" containerID="dc383cd6ffa56f54fc3c9a4d86b7eab89aaf977ed6f814179050c8b127a8d74a" exitCode=0 Sep 09 21:02:10.828980 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:10.828894 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerDied","Data":"dc383cd6ffa56f54fc3c9a4d86b7eab89aaf977ed6f814179050c8b127a8d74a"} Sep 09 21:02:11.307335 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:11.307301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:11.307489 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.307432 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:02:11.307531 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.307491 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:43.307473338 +0000 UTC m=+66.121323475 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:02:11.408333 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:11.408300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:11.408479 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:11.408358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:11.408516 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.408484 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:02:11.408516 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.408500 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:02:11.408516 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.408514 2578 projected.go:196] Error preparing data for projected volume kube-api-access-w6k97 for pod openshift-network-diagnostics/network-check-target-b6dd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:02:11.408621 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.408569 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97 podName:f018ab5a-68d5-42d0-9f15-2874b76a0ae1 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:43.408552335 +0000 UTC m=+66.222402479 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w6k97" (UniqueName: "kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97") pod "network-check-target-b6dd7" (UID: "f018ab5a-68d5-42d0-9f15-2874b76a0ae1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:02:11.409002 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.408983 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:02:11.409052 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:11.409038 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret podName:6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5 nodeName:}" failed. No retries permitted until 2026-09-09 21:02:43.409023149 +0000 UTC m=+66.222873294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret") pod "global-pull-secret-syncer-vcxpp" (UID: "6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:02:12.681576 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:12.681545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:12.682104 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:12.681560 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:12.682104 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:12.681664 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcxpp" podUID="6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5" Sep 09 21:02:12.682104 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:12.681684 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:12.682104 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:12.681778 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6dd7" podUID="f018ab5a-68d5-42d0-9f15-2874b76a0ae1" Sep 09 21:02:12.682104 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:12.681870 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-d9g6l" podUID="5e6ddbb9-c28f-44da-b932-5f722aab46f4" Sep 09 21:02:13.447514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.447486 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-18.ec2.internal" event="NodeReady" Sep 09 21:02:13.447667 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.447601 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 21:02:13.500745 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.500716 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zs7k6"] Sep 09 21:02:13.524946 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.524917 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lld6z"] Sep 09 21:02:13.525097 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.525036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.527753 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.527564 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 21:02:13.527753 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.527572 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 21:02:13.527753 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.527651 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9vjrs\"" Sep 09 21:02:13.540920 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.540899 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zs7k6"] Sep 09 21:02:13.541034 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.540939 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lld6z"] Sep 09 21:02:13.541034 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.541017 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:13.543978 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.543836 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 21:02:13.543978 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.543849 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4rmk\"" Sep 09 21:02:13.543978 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.543873 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 21:02:13.543978 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.543981 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 21:02:13.624460 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.624419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/90fcbf0d-a066-48c7-a685-4d51e021b67f-config-volume\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.624460 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.624466 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6951edbb-68bf-4b11-8bbc-33b960a5c4d5-cert\") pod \"ingress-canary-lld6z\" (UID: \"6951edbb-68bf-4b11-8bbc-33b960a5c4d5\") " pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:13.624689 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.624500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xqs\" (UniqueName: \"kubernetes.io/projected/90fcbf0d-a066-48c7-a685-4d51e021b67f-kube-api-access-d2xqs\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.624689 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.624524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jssb\" (UniqueName: \"kubernetes.io/projected/6951edbb-68bf-4b11-8bbc-33b960a5c4d5-kube-api-access-5jssb\") pod \"ingress-canary-lld6z\" (UID: \"6951edbb-68bf-4b11-8bbc-33b960a5c4d5\") " pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:13.624689 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.624569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90fcbf0d-a066-48c7-a685-4d51e021b67f-tmp-dir\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.624817 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.624698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.725995 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.725962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/90fcbf0d-a066-48c7-a685-4d51e021b67f-config-volume\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.726433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.726009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6951edbb-68bf-4b11-8bbc-33b960a5c4d5-cert\") pod \"ingress-canary-lld6z\" (UID: \"6951edbb-68bf-4b11-8bbc-33b960a5c4d5\") " pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:13.726433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.726154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xqs\" (UniqueName: \"kubernetes.io/projected/90fcbf0d-a066-48c7-a685-4d51e021b67f-kube-api-access-d2xqs\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.726433 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.726210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jssb\" (UniqueName: \"kubernetes.io/projected/6951edbb-68bf-4b11-8bbc-33b960a5c4d5-kube-api-access-5jssb\") pod \"ingress-canary-lld6z\" (UID: \"6951edbb-68bf-4b11-8bbc-33b960a5c4d5\") " pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:13.726563 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.726478 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90fcbf0d-a066-48c7-a685-4d51e021b67f-tmp-dir\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.726760 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.726595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.726810 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:13.726715 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:13.726843 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.726806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90fcbf0d-a066-48c7-a685-4d51e021b67f-tmp-dir\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.726843 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:13.726826 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls podName:90fcbf0d-a066-48c7-a685-4d51e021b67f nodeName:}" failed. No retries permitted until 2026-09-09 21:02:14.226812982 +0000 UTC m=+37.040663110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls") pod "dns-default-zs7k6" (UID: "90fcbf0d-a066-48c7-a685-4d51e021b67f") : secret "dns-default-metrics-tls" not found Sep 09 21:02:13.730732 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.730708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6951edbb-68bf-4b11-8bbc-33b960a5c4d5-cert\") pod \"ingress-canary-lld6z\" (UID: \"6951edbb-68bf-4b11-8bbc-33b960a5c4d5\") " pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:13.737065 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.737041 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xqs\" (UniqueName: \"kubernetes.io/projected/90fcbf0d-a066-48c7-a685-4d51e021b67f-kube-api-access-d2xqs\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.737376 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.737354 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jssb\" (UniqueName: \"kubernetes.io/projected/6951edbb-68bf-4b11-8bbc-33b960a5c4d5-kube-api-access-5jssb\") pod \"ingress-canary-lld6z\" (UID: \"6951edbb-68bf-4b11-8bbc-33b960a5c4d5\") " pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:13.737583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.737562 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/90fcbf0d-a066-48c7-a685-4d51e021b67f-config-volume\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:13.851353 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:13.851321 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lld6z" Sep 09 21:02:14.230316 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.230281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:14.230508 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:14.230453 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:14.230569 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:14.230522 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls podName:90fcbf0d-a066-48c7-a685-4d51e021b67f nodeName:}" failed. No retries permitted until 2026-09-09 21:02:15.230505063 +0000 UTC m=+38.044355219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls") pod "dns-default-zs7k6" (UID: "90fcbf0d-a066-48c7-a685-4d51e021b67f") : secret "dns-default-metrics-tls" not found Sep 09 21:02:14.682156 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.682061 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:14.682156 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.682090 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:14.682396 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.682074 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:14.685894 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.685867 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:02:14.685894 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.685880 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvxk\"" Sep 09 21:02:14.686087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.685937 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 21:02:14.686087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.685880 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 21:02:14.686087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.685881 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:02:14.686234 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:14.686148 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zv66x\"" Sep 09 21:02:15.237628 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:15.237601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:15.238121 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:15.237930 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:15.238121 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:15.238005 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls podName:90fcbf0d-a066-48c7-a685-4d51e021b67f nodeName:}" failed. No retries permitted until 2026-09-09 21:02:17.237980053 +0000 UTC m=+40.051830182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls") pod "dns-default-zs7k6" (UID: "90fcbf0d-a066-48c7-a685-4d51e021b67f") : secret "dns-default-metrics-tls" not found Sep 09 21:02:15.313264 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:15.313238 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lld6z"] Sep 09 21:02:15.316715 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:02:15.316688 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6951edbb_68bf_4b11_8bbc_33b960a5c4d5.slice/crio-b106f7dddd7aa052ea0613588d547da81190fff656f5ae4c7a5b1a5f58aad35e WatchSource:0}: Error finding container b106f7dddd7aa052ea0613588d547da81190fff656f5ae4c7a5b1a5f58aad35e: Status 404 returned error can't find the container with id b106f7dddd7aa052ea0613588d547da81190fff656f5ae4c7a5b1a5f58aad35e Sep 09 21:02:15.843723 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:15.843692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lld6z" event={"ID":"6951edbb-68bf-4b11-8bbc-33b960a5c4d5","Type":"ContainerStarted","Data":"b106f7dddd7aa052ea0613588d547da81190fff656f5ae4c7a5b1a5f58aad35e"} Sep 09 21:02:15.846630 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:15.846608 2578 generic.go:356] "Generic (PLEG): container finished" podID="00dbb388-31d8-4461-ba75-396770d7720e" containerID="0d398a1ad7f3f1401c98d254572603ba950ad66567e999edb676c595d8b3c9e8" exitCode=0 Sep 09 21:02:15.846748 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:15.846642 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerDied","Data":"0d398a1ad7f3f1401c98d254572603ba950ad66567e999edb676c595d8b3c9e8"} Sep 09 21:02:16.853495 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:16.853120 2578 generic.go:356] "Generic (PLEG): container finished" podID="00dbb388-31d8-4461-ba75-396770d7720e" containerID="31034237e2cdfd8e72d7eba82d52f9d0c03faab518c314bcf5e5dd7345201d55" exitCode=0 Sep 09 21:02:16.853495 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:16.853230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerDied","Data":"31034237e2cdfd8e72d7eba82d52f9d0c03faab518c314bcf5e5dd7345201d55"} Sep 09 21:02:17.250923 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:17.250886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:17.251078 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:17.251056 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:17.251131 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:17.251120 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls podName:90fcbf0d-a066-48c7-a685-4d51e021b67f nodeName:}" failed. No retries permitted until 2026-09-09 21:02:21.251105567 +0000 UTC m=+44.064955721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls") pod "dns-default-zs7k6" (UID: "90fcbf0d-a066-48c7-a685-4d51e021b67f") : secret "dns-default-metrics-tls" not found Sep 09 21:02:17.858765 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:17.858732 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-989mt" event={"ID":"00dbb388-31d8-4461-ba75-396770d7720e","Type":"ContainerStarted","Data":"6a02513807d0022679bd8848979507b1bca3a5bf3edeff2a803031889089cebf"} Sep 09 21:02:17.893338 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:17.893281 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-989mt" podStartSLOduration=8.664489178 podStartE2EDuration="39.893266528s" totalImagesPullingTime="31.22877735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:17.89096489 +0000 UTC m=+40.704815038" watchObservedRunningTime="2026-09-09 21:02:17.893266528 +0000 UTC m=+40.707116676" Sep 09 21:02:18.862552 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:18.862512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lld6z" event={"ID":"6951edbb-68bf-4b11-8bbc-33b960a5c4d5","Type":"ContainerStarted","Data":"8ce560f0d1982e738d5c35030a6527bc4b777900392b57139afe66f48aa2fca6"} Sep 09 21:02:18.883388 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:18.883345 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lld6z" podStartSLOduration=3.041170162 podStartE2EDuration="5.88332916s" totalImagesPullingTime="2.842158998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:18.882712148 +0000 UTC m=+41.696562297" watchObservedRunningTime="2026-09-09 21:02:18.88332916 +0000 UTC m=+41.697179308" Sep 09 21:02:21.278163 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:21.278112 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:21.278528 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:21.278276 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:21.278528 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:21.278347 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls podName:90fcbf0d-a066-48c7-a685-4d51e021b67f nodeName:}" failed. No retries permitted until 2026-09-09 21:02:29.278330567 +0000 UTC m=+52.092180694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls") pod "dns-default-zs7k6" (UID: "90fcbf0d-a066-48c7-a685-4d51e021b67f") : secret "dns-default-metrics-tls" not found Sep 09 21:02:29.333941 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:29.333885 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:29.334445 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:29.334049 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:29.334445 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:29.334111 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls podName:90fcbf0d-a066-48c7-a685-4d51e021b67f nodeName:}" failed. No retries permitted until 2026-09-09 21:02:45.334096802 +0000 UTC m=+68.147946945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls") pod "dns-default-zs7k6" (UID: "90fcbf0d-a066-48c7-a685-4d51e021b67f") : secret "dns-default-metrics-tls" not found Sep 09 21:02:34.206450 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.206413 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p"] Sep 09 21:02:34.208367 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.208348 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" Sep 09 21:02:34.210907 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.210880 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:02:34.211845 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.211820 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 09 21:02:34.211932 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.211919 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-z9qdj\"" Sep 09 21:02:34.212686 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.212669 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-647c9c8b45-hd7sg"] Sep 09 21:02:34.214371 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.214355 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.216960 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.216942 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 21:02:34.217055 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.216971 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 21:02:34.217055 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.217024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 21:02:34.217511 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.217493 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xcsfw\"" Sep 09 21:02:34.220639 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.220616 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p"] Sep 09 21:02:34.223339 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.223320 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 21:02:34.232445 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.232418 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-647c9c8b45-hd7sg"] Sep 09 21:02:34.317960 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.317934 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nzjj6"] Sep 09 21:02:34.319656 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.319643 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" Sep 09 21:02:34.322430 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.322414 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-s4cmm\"" Sep 09 21:02:34.335599 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.335577 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nzjj6"] Sep 09 21:02:34.367460 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-ca-trust-extracted\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367460 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-bound-sa-token\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367637 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-image-registry-private-configuration\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367637 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvk77\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-kube-api-access-jvk77\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367637 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367637 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-trusted-ca\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367779 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-installation-pull-secrets\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367779 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-certificates\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.367779 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.367689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqpzc\" (UniqueName: \"kubernetes.io/projected/0e64348c-0902-4b86-ad8e-1a2300b3a555-kube-api-access-mqpzc\") pod \"volume-data-source-validator-695c895bd6-vpp7p\" (UID: \"0e64348c-0902-4b86-ad8e-1a2300b3a555\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" Sep 09 21:02:34.417179 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.417150 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-2bt64"] Sep 09 21:02:34.419187 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.419172 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.422270 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.422238 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 09 21:02:34.422365 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.422314 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gltp5\"" Sep 09 21:02:34.422571 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.422556 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:02:34.422640 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.422586 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 09 21:02:34.424960 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.424940 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 09 21:02:34.428326 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.428310 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 09 21:02:34.433517 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.433499 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-2bt64"] Sep 09 21:02:34.468974 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.468958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469124 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.468985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-trusted-ca\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469124 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-installation-pull-secrets\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469124 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469054 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-certificates\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469124 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqpzc\" (UniqueName: \"kubernetes.io/projected/0e64348c-0902-4b86-ad8e-1a2300b3a555-kube-api-access-mqpzc\") pod \"volume-data-source-validator-695c895bd6-vpp7p\" (UID: \"0e64348c-0902-4b86-ad8e-1a2300b3a555\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" Sep 09 21:02:34.469124 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbflf\" (UniqueName: \"kubernetes.io/projected/55f1f42a-b939-41b0-a3ae-c53acf97d099-kube-api-access-dbflf\") pod \"network-check-source-685456655f-nzjj6\" (UID: \"55f1f42a-b939-41b0-a3ae-c53acf97d099\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" Sep 09 21:02:34.469386 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:34.469124 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:34.469386 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:34.469147 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-647c9c8b45-hd7sg: secret "image-registry-tls" not found Sep 09 21:02:34.469386 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-ca-trust-extracted\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469386 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:34.469247 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls podName:85d72fdd-c47c-4c25-9b74-26ce40bb0f6e nodeName:}" failed. No retries permitted until 2026-09-09 21:02:34.969222111 +0000 UTC m=+57.783072237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls") pod "image-registry-647c9c8b45-hd7sg" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e") : secret "image-registry-tls" not found Sep 09 21:02:34.469607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-bound-sa-token\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-image-registry-private-configuration\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469607 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvk77\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-kube-api-access-jvk77\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.469751 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.469625 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-ca-trust-extracted\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.470044 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.470009 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-certificates\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.470214 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.470178 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-trusted-ca\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.472476 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.472451 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-installation-pull-secrets\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.472569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.472487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-image-registry-private-configuration\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.478980 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.478962 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvk77\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-kube-api-access-jvk77\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.479180 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.479161 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqpzc\" (UniqueName: \"kubernetes.io/projected/0e64348c-0902-4b86-ad8e-1a2300b3a555-kube-api-access-mqpzc\") pod \"volume-data-source-validator-695c895bd6-vpp7p\" (UID: \"0e64348c-0902-4b86-ad8e-1a2300b3a555\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" Sep 09 21:02:34.479268 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.479247 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-bound-sa-token\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.516604 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.516583 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" Sep 09 21:02:34.570783 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.570753 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3063cd92-a3aa-4008-a6b1-90010561a7d3-trusted-ca\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.570895 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.570813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbflf\" (UniqueName: \"kubernetes.io/projected/55f1f42a-b939-41b0-a3ae-c53acf97d099-kube-api-access-dbflf\") pod \"network-check-source-685456655f-nzjj6\" (UID: \"55f1f42a-b939-41b0-a3ae-c53acf97d099\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" Sep 09 21:02:34.570965 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.570891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3063cd92-a3aa-4008-a6b1-90010561a7d3-config\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.570965 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.570947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3063cd92-a3aa-4008-a6b1-90010561a7d3-serving-cert\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.571138 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.570968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfx5k\" (UniqueName: \"kubernetes.io/projected/3063cd92-a3aa-4008-a6b1-90010561a7d3-kube-api-access-cfx5k\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.580406 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.580383 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbflf\" (UniqueName: \"kubernetes.io/projected/55f1f42a-b939-41b0-a3ae-c53acf97d099-kube-api-access-dbflf\") pod \"network-check-source-685456655f-nzjj6\" (UID: \"55f1f42a-b939-41b0-a3ae-c53acf97d099\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" Sep 09 21:02:34.627977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.627952 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" Sep 09 21:02:34.671977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.671810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3063cd92-a3aa-4008-a6b1-90010561a7d3-serving-cert\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.671977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.671871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfx5k\" (UniqueName: \"kubernetes.io/projected/3063cd92-a3aa-4008-a6b1-90010561a7d3-kube-api-access-cfx5k\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.672572 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.672448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3063cd92-a3aa-4008-a6b1-90010561a7d3-trusted-ca\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.672572 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.672562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3063cd92-a3aa-4008-a6b1-90010561a7d3-config\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.673971 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.673462 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3063cd92-a3aa-4008-a6b1-90010561a7d3-config\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.674036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.673961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3063cd92-a3aa-4008-a6b1-90010561a7d3-trusted-ca\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.675390 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.675371 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3063cd92-a3aa-4008-a6b1-90010561a7d3-serving-cert\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.682956 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.682929 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfx5k\" (UniqueName: \"kubernetes.io/projected/3063cd92-a3aa-4008-a6b1-90010561a7d3-kube-api-access-cfx5k\") pod \"console-operator-85b454874b-2bt64\" (UID: \"3063cd92-a3aa-4008-a6b1-90010561a7d3\") " pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.727846 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.727788 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:34.789553 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.789519 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p"] Sep 09 21:02:34.792714 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:02:34.792690 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e64348c_0902_4b86_ad8e_1a2300b3a555.slice/crio-a014cf2cef0fcdff52a93aa7a4c0bed2e4c5a08383dbfe9eebbd642e6373ff11 WatchSource:0}: Error finding container a014cf2cef0fcdff52a93aa7a4c0bed2e4c5a08383dbfe9eebbd642e6373ff11: Status 404 returned error can't find the container with id a014cf2cef0fcdff52a93aa7a4c0bed2e4c5a08383dbfe9eebbd642e6373ff11 Sep 09 21:02:34.900254 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.900220 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" event={"ID":"0e64348c-0902-4b86-ad8e-1a2300b3a555","Type":"ContainerStarted","Data":"a014cf2cef0fcdff52a93aa7a4c0bed2e4c5a08383dbfe9eebbd642e6373ff11"} Sep 09 21:02:34.902094 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.902069 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nzjj6"] Sep 09 21:02:34.904929 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:02:34.904907 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f1f42a_b939_41b0_a3ae_c53acf97d099.slice/crio-e2c503312a134a17d773def0c814077915470c953eeba89e78bfd2720bcad1f1 WatchSource:0}: Error finding container e2c503312a134a17d773def0c814077915470c953eeba89e78bfd2720bcad1f1: Status 404 returned error can't find the container with id e2c503312a134a17d773def0c814077915470c953eeba89e78bfd2720bcad1f1 Sep 09 21:02:34.974963 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:34.974933 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:34.975120 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:34.975101 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:34.975164 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:34.975122 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-647c9c8b45-hd7sg: secret "image-registry-tls" not found Sep 09 21:02:34.975221 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:34.975165 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls podName:85d72fdd-c47c-4c25-9b74-26ce40bb0f6e nodeName:}" failed. No retries permitted until 2026-09-09 21:02:35.975152598 +0000 UTC m=+58.789002729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls") pod "image-registry-647c9c8b45-hd7sg" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e") : secret "image-registry-tls" not found Sep 09 21:02:35.006618 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:35.006556 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-2bt64"] Sep 09 21:02:35.009444 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:02:35.009413 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3063cd92_a3aa_4008_a6b1_90010561a7d3.slice/crio-4dc4ee43039f5972a3a955466f034ed755728f3e4d405db57694c771fe71f034 WatchSource:0}: Error finding container 4dc4ee43039f5972a3a955466f034ed755728f3e4d405db57694c771fe71f034: Status 404 returned error can't find the container with id 4dc4ee43039f5972a3a955466f034ed755728f3e4d405db57694c771fe71f034 Sep 09 21:02:35.904939 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:35.904870 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2bt64" event={"ID":"3063cd92-a3aa-4008-a6b1-90010561a7d3","Type":"ContainerStarted","Data":"4dc4ee43039f5972a3a955466f034ed755728f3e4d405db57694c771fe71f034"} Sep 09 21:02:35.906465 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:35.906422 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" event={"ID":"55f1f42a-b939-41b0-a3ae-c53acf97d099","Type":"ContainerStarted","Data":"e2c503312a134a17d773def0c814077915470c953eeba89e78bfd2720bcad1f1"} Sep 09 21:02:35.982499 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:35.981959 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:35.982499 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:35.982106 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:35.982499 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:35.982123 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-647c9c8b45-hd7sg: secret "image-registry-tls" not found Sep 09 21:02:35.982499 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:35.982185 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls podName:85d72fdd-c47c-4c25-9b74-26ce40bb0f6e nodeName:}" failed. No retries permitted until 2026-09-09 21:02:37.982164976 +0000 UTC m=+60.796015127 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls") pod "image-registry-647c9c8b45-hd7sg" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e") : secret "image-registry-tls" not found Sep 09 21:02:36.910854 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:36.910816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" event={"ID":"0e64348c-0902-4b86-ad8e-1a2300b3a555","Type":"ContainerStarted","Data":"dfe8b17f0d3e3c0ab67f5cfbb1ee3f168a490f3f18410a1c277155947495c636"} Sep 09 21:02:36.930321 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:36.930268 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-vpp7p" podStartSLOduration=1.2848831459999999 podStartE2EDuration="2.930246053s" totalImagesPullingTime="1.645362907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:36.929117276 +0000 UTC m=+59.742967452" watchObservedRunningTime="2026-09-09 21:02:36.930246053 +0000 UTC m=+59.744096204" Sep 09 21:02:37.840566 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:37.840534 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-chpbt" Sep 09 21:02:37.999642 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:37.999602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:38.000049 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:37.999750 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:38.000049 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:37.999765 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-647c9c8b45-hd7sg: secret "image-registry-tls" not found Sep 09 21:02:38.000049 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:37.999831 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls podName:85d72fdd-c47c-4c25-9b74-26ce40bb0f6e nodeName:}" failed. No retries permitted until 2026-09-09 21:02:41.999811338 +0000 UTC m=+64.813661471 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls") pod "image-registry-647c9c8b45-hd7sg" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e") : secret "image-registry-tls" not found Sep 09 21:02:38.917783 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:38.917757 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/0.log" Sep 09 21:02:38.917897 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:38.917801 2578 generic.go:356] "Generic (PLEG): container finished" podID="3063cd92-a3aa-4008-a6b1-90010561a7d3" containerID="5907e86adf84add62fb15dbe8161665216093751e44006decd01d33e11666814" exitCode=255 Sep 09 21:02:38.917897 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:38.917883 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2bt64" event={"ID":"3063cd92-a3aa-4008-a6b1-90010561a7d3","Type":"ContainerDied","Data":"5907e86adf84add62fb15dbe8161665216093751e44006decd01d33e11666814"} Sep 09 21:02:38.918116 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:38.918101 2578 scope.go:117] "RemoveContainer" containerID="5907e86adf84add62fb15dbe8161665216093751e44006decd01d33e11666814" Sep 09 21:02:38.919611 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:38.919592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" event={"ID":"55f1f42a-b939-41b0-a3ae-c53acf97d099","Type":"ContainerStarted","Data":"76c5fe6cd2e2975dacb57e9c4cacffda97c39d3c9eef5b0390d7cf3cec292a48"} Sep 09 21:02:38.974064 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:38.974023 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-nzjj6" podStartSLOduration=1.101168225 podStartE2EDuration="4.974010164s" totalImagesPullingTime="3.872841939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:38.973555404 +0000 UTC m=+61.787405555" watchObservedRunningTime="2026-09-09 21:02:38.974010164 +0000 UTC m=+61.787860313" Sep 09 21:02:39.043407 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.043378 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf"] Sep 09 21:02:39.045738 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.045718 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" Sep 09 21:02:39.050915 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.050895 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 09 21:02:39.051008 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.050891 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 09 21:02:39.054183 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.054167 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-chhb8\"" Sep 09 21:02:39.080889 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.080863 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf"] Sep 09 21:02:39.106691 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.106668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj4ns\" (UniqueName: \"kubernetes.io/projected/366c43e6-0530-4f81-a877-a726708279cd-kube-api-access-mj4ns\") pod \"migrator-fdbb8fdff-xptdf\" (UID: \"366c43e6-0530-4f81-a877-a726708279cd\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" Sep 09 21:02:39.207137 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.207108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj4ns\" (UniqueName: \"kubernetes.io/projected/366c43e6-0530-4f81-a877-a726708279cd-kube-api-access-mj4ns\") pod \"migrator-fdbb8fdff-xptdf\" (UID: \"366c43e6-0530-4f81-a877-a726708279cd\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" Sep 09 21:02:39.221734 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.221709 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj4ns\" (UniqueName: \"kubernetes.io/projected/366c43e6-0530-4f81-a877-a726708279cd-kube-api-access-mj4ns\") pod \"migrator-fdbb8fdff-xptdf\" (UID: \"366c43e6-0530-4f81-a877-a726708279cd\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" Sep 09 21:02:39.415019 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.414937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" Sep 09 21:02:39.567373 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.567339 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bxz7h"] Sep 09 21:02:39.569505 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.569490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.571950 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.571926 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 21:02:39.572290 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.572265 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 21:02:39.572519 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.572502 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 21:02:39.572519 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.572514 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 21:02:39.573104 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.573087 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2hhmg\"" Sep 09 21:02:39.589593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.589568 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bxz7h"] Sep 09 21:02:39.610326 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.610305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.610427 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.610365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.610427 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.610419 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-data-volume\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.610512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.610443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-crio-socket\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.610512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.610462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjct\" (UniqueName: \"kubernetes.io/projected/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-api-access-dsjct\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.694809 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.694739 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf"] Sep 09 21:02:39.697395 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:02:39.697373 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod366c43e6_0530_4f81_a877_a726708279cd.slice/crio-4b5a9120c941cd7a8c0df13659803a0a02ce1d09932b24aafaac3daf44d38c26 WatchSource:0}: Error finding container 4b5a9120c941cd7a8c0df13659803a0a02ce1d09932b24aafaac3daf44d38c26: Status 404 returned error can't find the container with id 4b5a9120c941cd7a8c0df13659803a0a02ce1d09932b24aafaac3daf44d38c26 Sep 09 21:02:39.711684 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.711659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-crio-socket\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.711767 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.711688 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjct\" (UniqueName: \"kubernetes.io/projected/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-api-access-dsjct\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.711849 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.711792 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-crio-socket\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.711884 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.711851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.711930 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.711908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.711964 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.711947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-data-volume\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.712013 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:39.711999 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:39.712064 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:39.712054 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:40.212040057 +0000 UTC m=+63.025890184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : configmap "kube-rbac-proxy" not found Sep 09 21:02:39.712117 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:39.711999 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:39.712159 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:39.712144 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:40.212129957 +0000 UTC m=+63.025980085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:39.712307 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.712292 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-data-volume\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.722168 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.722150 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjct\" (UniqueName: \"kubernetes.io/projected/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-api-access-dsjct\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:39.923946 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.923920 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:02:39.924368 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.924350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/0.log" Sep 09 21:02:39.924481 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.924392 2578 generic.go:356] "Generic (PLEG): container finished" podID="3063cd92-a3aa-4008-a6b1-90010561a7d3" containerID="fe3fbf66edc374102456de7f94e33906b2690291912433fd3b8051385d99fe5d" exitCode=255 Sep 09 21:02:39.924541 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.924474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2bt64" event={"ID":"3063cd92-a3aa-4008-a6b1-90010561a7d3","Type":"ContainerDied","Data":"fe3fbf66edc374102456de7f94e33906b2690291912433fd3b8051385d99fe5d"} Sep 09 21:02:39.924541 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.924530 2578 scope.go:117] "RemoveContainer" containerID="5907e86adf84add62fb15dbe8161665216093751e44006decd01d33e11666814" Sep 09 21:02:39.924667 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.924651 2578 scope.go:117] "RemoveContainer" containerID="fe3fbf66edc374102456de7f94e33906b2690291912433fd3b8051385d99fe5d" Sep 09 21:02:39.924841 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:39.924819 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-2bt64_openshift-console-operator(3063cd92-a3aa-4008-a6b1-90010561a7d3)\"" pod="openshift-console-operator/console-operator-85b454874b-2bt64" podUID="3063cd92-a3aa-4008-a6b1-90010561a7d3" Sep 09 21:02:39.925905 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:39.925760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" event={"ID":"366c43e6-0530-4f81-a877-a726708279cd","Type":"ContainerStarted","Data":"4b5a9120c941cd7a8c0df13659803a0a02ce1d09932b24aafaac3daf44d38c26"} Sep 09 21:02:40.215913 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:40.215881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:40.216314 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:40.215927 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:40.216314 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:40.216015 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:40.216314 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:40.216023 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:40.216314 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:40.216083 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:41.216058727 +0000 UTC m=+64.029908857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : configmap "kube-rbac-proxy" not found Sep 09 21:02:40.216314 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:40.216104 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:41.216093545 +0000 UTC m=+64.029943672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:40.930013 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:40.929987 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:02:40.930364 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:40.930347 2578 scope.go:117] "RemoveContainer" containerID="fe3fbf66edc374102456de7f94e33906b2690291912433fd3b8051385d99fe5d" Sep 09 21:02:40.930532 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:40.930512 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-2bt64_openshift-console-operator(3063cd92-a3aa-4008-a6b1-90010561a7d3)\"" pod="openshift-console-operator/console-operator-85b454874b-2bt64" podUID="3063cd92-a3aa-4008-a6b1-90010561a7d3" Sep 09 21:02:41.223064 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:41.223041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:41.223494 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:41.223088 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:41.223494 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:41.223161 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:41.223494 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:41.223225 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:43.223211768 +0000 UTC m=+66.037061908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : configmap "kube-rbac-proxy" not found Sep 09 21:02:41.223494 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:41.223237 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:41.223494 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:41.223297 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:43.223280529 +0000 UTC m=+66.037130655 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:42.028666 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:42.028620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:42.028832 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:42.028765 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:42.028832 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:42.028782 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-647c9c8b45-hd7sg: secret "image-registry-tls" not found Sep 09 21:02:42.028832 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:42.028833 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls podName:85d72fdd-c47c-4c25-9b74-26ce40bb0f6e nodeName:}" failed. No retries permitted until 2026-09-09 21:02:50.028818041 +0000 UTC m=+72.842668185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls") pod "image-registry-647c9c8b45-hd7sg" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e") : secret "image-registry-tls" not found Sep 09 21:02:42.537633 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:42.537599 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjhvd_7f2d556d-c2dc-4e5f-b243-b320d11c24d4/dns-node-resolver/0.log" Sep 09 21:02:42.937998 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:42.937918 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" event={"ID":"366c43e6-0530-4f81-a877-a726708279cd","Type":"ContainerStarted","Data":"f9f11961156c0329f5e5cce9070b3f25d5c6b8ec5d65e8ee6aaf5ea14545b411"} Sep 09 21:02:42.937998 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:42.937994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" event={"ID":"366c43e6-0530-4f81-a877-a726708279cd","Type":"ContainerStarted","Data":"9943e0d679256380bff729581eb1344fd07e96b7a485ee2fadee94bd5b4e4863"} Sep 09 21:02:42.959075 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:42.959030 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-xptdf" podStartSLOduration=1.082804623 podStartE2EDuration="3.959016523s" totalImagesPullingTime="2.8762119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:42.958022535 +0000 UTC m=+65.771872684" watchObservedRunningTime="2026-09-09 21:02:42.959016523 +0000 UTC m=+65.772866672" Sep 09 21:02:43.137021 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.136993 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7lm7d_85e4a129-3ede-4946-b4a2-2d86b351df38/node-ca/0.log" Sep 09 21:02:43.237150 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.237124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:43.237301 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.237172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:43.237301 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:43.237275 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:43.237301 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:43.237295 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:43.237422 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:43.237332 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:47.237318315 +0000 UTC m=+70.051168446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : configmap "kube-rbac-proxy" not found Sep 09 21:02:43.237422 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:43.237345 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:47.237339875 +0000 UTC m=+70.051190001 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:43.338017 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.337989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:02:43.340708 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.340692 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:02:43.348749 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:43.348731 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 09 21:02:43.348804 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:43.348777 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs podName:5e6ddbb9-c28f-44da-b932-5f722aab46f4 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:47.348764549 +0000 UTC m=+130.162614676 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs") pod "network-metrics-daemon-d9g6l" (UID: "5e6ddbb9-c28f-44da-b932-5f722aab46f4") : secret "metrics-daemon-secret" not found Sep 09 21:02:43.438954 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.438924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:43.439065 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.438975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:43.441710 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.441680 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6k97\" (UniqueName: \"kubernetes.io/projected/f018ab5a-68d5-42d0-9f15-2874b76a0ae1-kube-api-access-w6k97\") pod \"network-check-target-b6dd7\" (UID: \"f018ab5a-68d5-42d0-9f15-2874b76a0ae1\") " pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:43.443227 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.443212 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:02:43.451271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.451253 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5-original-pull-secret\") pod \"global-pull-secret-syncer-vcxpp\" (UID: \"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5\") " pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:43.495157 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.495102 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zv66x\"" Sep 09 21:02:43.501216 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.501184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcxpp" Sep 09 21:02:43.503092 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.503079 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:43.776561 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.776487 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6dd7"] Sep 09 21:02:43.779158 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.779098 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcxpp"] Sep 09 21:02:43.780213 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:02:43.780170 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf018ab5a_68d5_42d0_9f15_2874b76a0ae1.slice/crio-2a2fb49436be8ec1176fbb9d046a710963afd7d170971e87ff8f9f9030fd20d0 WatchSource:0}: Error finding container 2a2fb49436be8ec1176fbb9d046a710963afd7d170971e87ff8f9f9030fd20d0: Status 404 returned error can't find the container with id 2a2fb49436be8ec1176fbb9d046a710963afd7d170971e87ff8f9f9030fd20d0 Sep 09 21:02:43.781801 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:02:43.781782 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c202f57_bbf0_4eef_8fbc_8c8f2dd107a5.slice/crio-9ce77a06a366b29cdb98366696e6a8bfd2e6efa3edb36bf66d3bb113d85c6246 WatchSource:0}: Error finding container 9ce77a06a366b29cdb98366696e6a8bfd2e6efa3edb36bf66d3bb113d85c6246: Status 404 returned error can't find the container with id 9ce77a06a366b29cdb98366696e6a8bfd2e6efa3edb36bf66d3bb113d85c6246 Sep 09 21:02:43.942795 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.942751 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6dd7" event={"ID":"f018ab5a-68d5-42d0-9f15-2874b76a0ae1","Type":"ContainerStarted","Data":"c771fa46fc7321dba53934557e5c70e7e2e02c6fcc1c8730a700d1720bbaf392"} Sep 09 21:02:43.942959 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.942808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6dd7" event={"ID":"f018ab5a-68d5-42d0-9f15-2874b76a0ae1","Type":"ContainerStarted","Data":"2a2fb49436be8ec1176fbb9d046a710963afd7d170971e87ff8f9f9030fd20d0"} Sep 09 21:02:43.942959 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.942834 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:02:43.943927 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.943900 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcxpp" event={"ID":"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5","Type":"ContainerStarted","Data":"9ce77a06a366b29cdb98366696e6a8bfd2e6efa3edb36bf66d3bb113d85c6246"} Sep 09 21:02:43.965849 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:43.965793 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b6dd7" podStartSLOduration=65.965777722 podStartE2EDuration="1m5.965777722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:43.965368835 +0000 UTC m=+66.779218983" watchObservedRunningTime="2026-09-09 21:02:43.965777722 +0000 UTC m=+66.779627873" Sep 09 21:02:44.142338 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:44.142271 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lld6z_6951edbb-68bf-4b11-8bbc-33b960a5c4d5/serve-healthcheck-canary/0.log" Sep 09 21:02:44.538892 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:44.538860 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-xptdf_366c43e6-0530-4f81-a877-a726708279cd/migrator/0.log" Sep 09 21:02:44.728052 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:44.728024 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:44.728265 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:44.728122 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:02:44.728471 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:44.728455 2578 scope.go:117] "RemoveContainer" containerID="fe3fbf66edc374102456de7f94e33906b2690291912433fd3b8051385d99fe5d" Sep 09 21:02:44.728646 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:44.728626 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-2bt64_openshift-console-operator(3063cd92-a3aa-4008-a6b1-90010561a7d3)\"" pod="openshift-console-operator/console-operator-85b454874b-2bt64" podUID="3063cd92-a3aa-4008-a6b1-90010561a7d3" Sep 09 21:02:44.739924 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:44.739902 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-xptdf_366c43e6-0530-4f81-a877-a726708279cd/graceful-termination/0.log" Sep 09 21:02:44.948093 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:44.948016 2578 scope.go:117] "RemoveContainer" containerID="fe3fbf66edc374102456de7f94e33906b2690291912433fd3b8051385d99fe5d" Sep 09 21:02:44.948493 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:44.948241 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-2bt64_openshift-console-operator(3063cd92-a3aa-4008-a6b1-90010561a7d3)\"" pod="openshift-console-operator/console-operator-85b454874b-2bt64" podUID="3063cd92-a3aa-4008-a6b1-90010561a7d3" Sep 09 21:02:45.353343 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:45.353300 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:02:45.353529 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:45.353460 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:02:45.353597 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:45.353539 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls podName:90fcbf0d-a066-48c7-a685-4d51e021b67f nodeName:}" failed. No retries permitted until 2026-09-09 21:03:17.353517934 +0000 UTC m=+100.167368087 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls") pod "dns-default-zs7k6" (UID: "90fcbf0d-a066-48c7-a685-4d51e021b67f") : secret "dns-default-metrics-tls" not found Sep 09 21:02:47.271473 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:47.271428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:47.271907 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:47.271489 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:47.271907 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:47.271612 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:02:47.271907 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:47.271614 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:47.271907 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:47.271686 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:55.271666009 +0000 UTC m=+78.085516166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : configmap "kube-rbac-proxy" not found Sep 09 21:02:47.271907 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:47.271704 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:02:55.27169519 +0000 UTC m=+78.085545322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:47.958308 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:47.958273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcxpp" event={"ID":"6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5","Type":"ContainerStarted","Data":"9599fd8cf8b941a7114476fefd8f16e4fdca309562248fb8c42cb179949fd9b0"} Sep 09 21:02:47.993583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:47.993540 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vcxpp" podStartSLOduration=66.389203642 podStartE2EDuration="1m9.99352699s" totalImagesPullingTime="3.604323348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:47.993209068 +0000 UTC m=+70.807059209" watchObservedRunningTime="2026-09-09 21:02:47.99352699 +0000 UTC m=+70.807377139" Sep 09 21:02:50.093079 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:50.093044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:02:50.093481 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:50.093160 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:02:50.093481 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:50.093171 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-647c9c8b45-hd7sg: secret "image-registry-tls" not found Sep 09 21:02:50.093481 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:50.093239 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls podName:85d72fdd-c47c-4c25-9b74-26ce40bb0f6e nodeName:}" failed. No retries permitted until 2026-09-09 21:03:06.093222672 +0000 UTC m=+88.907072818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls") pod "image-registry-647c9c8b45-hd7sg" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e") : secret "image-registry-tls" not found Sep 09 21:02:55.332837 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:55.332795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:55.333253 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:55.332847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:55.333253 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:55.332943 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:02:55.333253 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:02:55.333028 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls podName:0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa nodeName:}" failed. No retries permitted until 2026-09-09 21:03:11.333012551 +0000 UTC m=+94.146862678 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bxz7h" (UID: "0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa") : secret "insights-runtime-extractor-tls" not found Sep 09 21:02:55.333395 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:55.333379 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:02:59.682431 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:59.682325 2578 scope.go:117] "RemoveContainer" containerID="fe3fbf66edc374102456de7f94e33906b2690291912433fd3b8051385d99fe5d" Sep 09 21:02:59.999681 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:59.999650 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:02:59.999815 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:59.999707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-2bt64" event={"ID":"3063cd92-a3aa-4008-a6b1-90010561a7d3","Type":"ContainerStarted","Data":"c06787ef0124b849d36114707a0860fb53fe3f6200f9f5f14ab9f5de41e2d957"} Sep 09 21:03:00.000008 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:02:59.999981 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:03:00.025776 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:00.025720 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-2bt64" podStartSLOduration=22.261347073 podStartE2EDuration="26.025701714s" totalImagesPullingTime="3.764354641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:00.02472241 +0000 UTC m=+82.838572560" watchObservedRunningTime="2026-09-09 21:03:00.025701714 +0000 UTC m=+82.839551865" Sep 09 21:03:00.378354 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:00.378274 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-2bt64" Sep 09 21:03:06.104740 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:06.104709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:03:06.107281 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:06.107258 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"image-registry-647c9c8b45-hd7sg\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:03:06.326960 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:06.326927 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xcsfw\"" Sep 09 21:03:06.335028 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:06.335008 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:03:06.611107 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:06.611086 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-647c9c8b45-hd7sg"] Sep 09 21:03:06.613531 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:06.613502 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d72fdd_c47c_4c25_9b74_26ce40bb0f6e.slice/crio-9edf63ba960176021e2514b5ebe9099d49ce3cdb5745dcd4a120dface24de428 WatchSource:0}: Error finding container 9edf63ba960176021e2514b5ebe9099d49ce3cdb5745dcd4a120dface24de428: Status 404 returned error can't find the container with id 9edf63ba960176021e2514b5ebe9099d49ce3cdb5745dcd4a120dface24de428 Sep 09 21:03:07.023319 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:07.023280 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" event={"ID":"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e","Type":"ContainerStarted","Data":"f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827"} Sep 09 21:03:07.023467 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:07.023374 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" event={"ID":"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e","Type":"ContainerStarted","Data":"9edf63ba960176021e2514b5ebe9099d49ce3cdb5745dcd4a120dface24de428"} Sep 09 21:03:07.023467 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:07.023433 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:03:07.063916 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:07.063875 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" podStartSLOduration=33.063862432 podStartE2EDuration="33.063862432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:07.062499885 +0000 UTC m=+89.876350033" watchObservedRunningTime="2026-09-09 21:03:07.063862432 +0000 UTC m=+89.877712578" Sep 09 21:03:11.347262 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:11.347222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:03:11.349561 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:11.349543 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bxz7h\" (UID: \"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa\") " pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:03:11.378556 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:11.378529 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bxz7h" Sep 09 21:03:11.671818 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:11.671747 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bxz7h"] Sep 09 21:03:11.675283 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:11.675258 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0973e706_c8f1_4cdb_b5c2_69ce3a5b77fa.slice/crio-d70f4d6b83f748d1003dc1d8ace435df5fefbc8b7b56db13c25cf4d6f5b8907a WatchSource:0}: Error finding container d70f4d6b83f748d1003dc1d8ace435df5fefbc8b7b56db13c25cf4d6f5b8907a: Status 404 returned error can't find the container with id d70f4d6b83f748d1003dc1d8ace435df5fefbc8b7b56db13c25cf4d6f5b8907a Sep 09 21:03:12.043634 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:12.043605 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxz7h" event={"ID":"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa","Type":"ContainerStarted","Data":"5762283cd878610b340b377db60bdbb60dfd57f904ec75ca151c70d52f606235"} Sep 09 21:03:12.043770 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:12.043659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxz7h" event={"ID":"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa","Type":"ContainerStarted","Data":"d70f4d6b83f748d1003dc1d8ace435df5fefbc8b7b56db13c25cf4d6f5b8907a"} Sep 09 21:03:14.052907 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:14.052873 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxz7h" event={"ID":"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa","Type":"ContainerStarted","Data":"5a1ffeefe897f660b6594c1776ba77d1407475358ebe7aa722c07738faf6b9f8"} Sep 09 21:03:14.949496 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:14.949474 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b6dd7" Sep 09 21:03:15.059918 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:15.059874 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bxz7h" event={"ID":"0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa","Type":"ContainerStarted","Data":"5122190b49c1c858fe53029c8478ee017950e54f58733cc35bb6b0042c2f9937"} Sep 09 21:03:15.085372 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:15.085325 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bxz7h" podStartSLOduration=33.041297057 podStartE2EDuration="36.085310605s" totalImagesPullingTime="3.044013548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:02:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:15.085246437 +0000 UTC m=+97.899096585" watchObservedRunningTime="2026-09-09 21:03:15.085310605 +0000 UTC m=+97.899160754" Sep 09 21:03:17.392672 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:17.392627 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:03:17.395020 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:17.395000 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/90fcbf0d-a066-48c7-a685-4d51e021b67f-metrics-tls\") pod \"dns-default-zs7k6\" (UID: \"90fcbf0d-a066-48c7-a685-4d51e021b67f\") " pod="openshift-dns/dns-default-zs7k6" Sep 09 21:03:17.438869 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:17.438844 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9vjrs\"" Sep 09 21:03:17.445749 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:17.445731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zs7k6" Sep 09 21:03:17.725292 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:17.725165 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zs7k6"] Sep 09 21:03:17.727721 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:17.727698 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90fcbf0d_a066_48c7_a685_4d51e021b67f.slice/crio-3b2bfd27d2a6fd08d2813c97394be36f319f0385df3e40e2a593abfe88302a4b WatchSource:0}: Error finding container 3b2bfd27d2a6fd08d2813c97394be36f319f0385df3e40e2a593abfe88302a4b: Status 404 returned error can't find the container with id 3b2bfd27d2a6fd08d2813c97394be36f319f0385df3e40e2a593abfe88302a4b Sep 09 21:03:18.069725 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:18.069653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zs7k6" event={"ID":"90fcbf0d-a066-48c7-a685-4d51e021b67f","Type":"ContainerStarted","Data":"3b2bfd27d2a6fd08d2813c97394be36f319f0385df3e40e2a593abfe88302a4b"} Sep 09 21:03:21.082652 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.082612 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zs7k6" event={"ID":"90fcbf0d-a066-48c7-a685-4d51e021b67f","Type":"ContainerStarted","Data":"71b30e949b9e757358d6f3da5454a9865ff60f9dcde83f879565a7c36d5d6c99"} Sep 09 21:03:21.083101 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.082671 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zs7k6" event={"ID":"90fcbf0d-a066-48c7-a685-4d51e021b67f","Type":"ContainerStarted","Data":"3dd84c224fd7d17b0a723b10d11dae97a395b69d7541b85f5c2c7ad866a1e250"} Sep 09 21:03:21.083101 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.082730 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zs7k6" Sep 09 21:03:21.106692 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.106648 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zs7k6" podStartSLOduration=65.830643988 podStartE2EDuration="1m8.106633314s" totalImagesPullingTime="2.275989326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:21.106298067 +0000 UTC m=+103.920148217" watchObservedRunningTime="2026-09-09 21:03:21.106633314 +0000 UTC m=+103.920483463" Sep 09 21:03:21.140477 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.140451 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6"] Sep 09 21:03:21.142638 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.142624 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.144885 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.144863 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 21:03:21.144986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.144931 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kqlpl\"" Sep 09 21:03:21.145061 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.144984 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 21:03:21.145248 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.145230 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 21:03:21.145358 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.145340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 09 21:03:21.160573 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.160549 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6"] Sep 09 21:03:21.220669 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.220626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cff4cd5-a2b8-4871-8f70-37a239f688ce-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-574d9f77bd-nngl6\" (UID: \"2cff4cd5-a2b8-4871-8f70-37a239f688ce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.220841 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.220762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcm6n\" (UniqueName: \"kubernetes.io/projected/2cff4cd5-a2b8-4871-8f70-37a239f688ce-kube-api-access-hcm6n\") pod \"managed-serviceaccount-addon-agent-574d9f77bd-nngl6\" (UID: \"2cff4cd5-a2b8-4871-8f70-37a239f688ce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.299311 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.299285 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f"] Sep 09 21:03:21.301311 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.301285 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" Sep 09 21:03:21.304094 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.304078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 09 21:03:21.304576 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.304557 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-68h5s\"" Sep 09 21:03:21.305650 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.305632 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-647c9c8b45-hd7sg"] Sep 09 21:03:21.310282 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.310259 2578 patch_prober.go:28] interesting pod/image-registry-647c9c8b45-hd7sg container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:03:21.310376 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.310302 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" podUID="85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:03:21.322079 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.322058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcm6n\" (UniqueName: \"kubernetes.io/projected/2cff4cd5-a2b8-4871-8f70-37a239f688ce-kube-api-access-hcm6n\") pod \"managed-serviceaccount-addon-agent-574d9f77bd-nngl6\" (UID: \"2cff4cd5-a2b8-4871-8f70-37a239f688ce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.322235 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.322103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cff4cd5-a2b8-4871-8f70-37a239f688ce-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-574d9f77bd-nngl6\" (UID: \"2cff4cd5-a2b8-4871-8f70-37a239f688ce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.324648 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.324624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cff4cd5-a2b8-4871-8f70-37a239f688ce-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-574d9f77bd-nngl6\" (UID: \"2cff4cd5-a2b8-4871-8f70-37a239f688ce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.331632 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.331612 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f"] Sep 09 21:03:21.341889 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.341836 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcm6n\" (UniqueName: \"kubernetes.io/projected/2cff4cd5-a2b8-4871-8f70-37a239f688ce-kube-api-access-hcm6n\") pod \"managed-serviceaccount-addon-agent-574d9f77bd-nngl6\" (UID: \"2cff4cd5-a2b8-4871-8f70-37a239f688ce\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.396540 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.396512 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-787f6d466-dsx87"] Sep 09 21:03:21.398432 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.398411 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.422665 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.422647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e89d8e1d-d6fc-4236-8b4b-de98a07b1555-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-6fp9f\" (UID: \"e89d8e1d-d6fc-4236-8b4b-de98a07b1555\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" Sep 09 21:03:21.463830 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.463807 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" Sep 09 21:03:21.486077 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.486049 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787f6d466-dsx87"] Sep 09 21:03:21.523744 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.523716 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-trusted-ca\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.523855 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.523763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-image-registry-private-configuration\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.523896 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.523872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-registry-tls\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.523932 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.523911 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-registry-certificates\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.524024 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.524003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-ca-trust-extracted\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.524086 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.524041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-bound-sa-token\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.524086 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.524072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lptq5\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-kube-api-access-lptq5\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.524220 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.524116 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-installation-pull-secrets\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.524220 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.524180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e89d8e1d-d6fc-4236-8b4b-de98a07b1555-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-6fp9f\" (UID: \"e89d8e1d-d6fc-4236-8b4b-de98a07b1555\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" Sep 09 21:03:21.526648 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.526624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e89d8e1d-d6fc-4236-8b4b-de98a07b1555-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-6fp9f\" (UID: \"e89d8e1d-d6fc-4236-8b4b-de98a07b1555\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" Sep 09 21:03:21.611273 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.611180 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" Sep 09 21:03:21.624743 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.624718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-registry-tls\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.624852 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.624750 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-registry-certificates\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.624852 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.624778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-ca-trust-extracted\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.624852 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.624794 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-bound-sa-token\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.624852 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.624810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lptq5\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-kube-api-access-lptq5\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.625049 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.624935 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-installation-pull-secrets\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.625049 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.625020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-trusted-ca\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.625124 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.625061 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-image-registry-private-configuration\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.625235 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.625183 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-ca-trust-extracted\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.626076 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.626055 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-registry-certificates\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.626236 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.626215 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-trusted-ca\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.627587 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.627564 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-registry-tls\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.628040 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.628022 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-image-registry-private-configuration\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.628116 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.628041 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-installation-pull-secrets\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.639747 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.639716 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-bound-sa-token\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.652039 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.652014 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lptq5\" (UniqueName: \"kubernetes.io/projected/4f960ddd-7f52-43b4-a121-7a14d7d13e9f-kube-api-access-lptq5\") pod \"image-registry-787f6d466-dsx87\" (UID: \"4f960ddd-7f52-43b4-a121-7a14d7d13e9f\") " pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.707077 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.707049 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:21.745727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.745698 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6"] Sep 09 21:03:21.748716 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:21.748687 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cff4cd5_a2b8_4871_8f70_37a239f688ce.slice/crio-b35f4edccad8b014ec7cd1e5971dcb658af39c429c7ea220edffc8b8dbbf8a7f WatchSource:0}: Error finding container b35f4edccad8b014ec7cd1e5971dcb658af39c429c7ea220edffc8b8dbbf8a7f: Status 404 returned error can't find the container with id b35f4edccad8b014ec7cd1e5971dcb658af39c429c7ea220edffc8b8dbbf8a7f Sep 09 21:03:21.884740 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.884567 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f"] Sep 09 21:03:21.886988 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:21.886963 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode89d8e1d_d6fc_4236_8b4b_de98a07b1555.slice/crio-70dfc0e31eaed28b7d36cd58f1bf7728c6e8f99b1e7f34daed77e64e2a6baebf WatchSource:0}: Error finding container 70dfc0e31eaed28b7d36cd58f1bf7728c6e8f99b1e7f34daed77e64e2a6baebf: Status 404 returned error can't find the container with id 70dfc0e31eaed28b7d36cd58f1bf7728c6e8f99b1e7f34daed77e64e2a6baebf Sep 09 21:03:21.986228 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:21.986207 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787f6d466-dsx87"] Sep 09 21:03:21.988548 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:21.988522 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f960ddd_7f52_43b4_a121_7a14d7d13e9f.slice/crio-0a46de848c89d4dbb7fbd1275149b71de51a9b08f2e50cde86a944f724624233 WatchSource:0}: Error finding container 0a46de848c89d4dbb7fbd1275149b71de51a9b08f2e50cde86a944f724624233: Status 404 returned error can't find the container with id 0a46de848c89d4dbb7fbd1275149b71de51a9b08f2e50cde86a944f724624233 Sep 09 21:03:22.092085 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:22.092049 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787f6d466-dsx87" event={"ID":"4f960ddd-7f52-43b4-a121-7a14d7d13e9f","Type":"ContainerStarted","Data":"314d0191988fc5e3c4b5e25b2495fe0b39323dbcf001b00c1872137f01c564a6"} Sep 09 21:03:22.092503 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:22.092147 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:22.092503 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:22.092178 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787f6d466-dsx87" event={"ID":"4f960ddd-7f52-43b4-a121-7a14d7d13e9f","Type":"ContainerStarted","Data":"0a46de848c89d4dbb7fbd1275149b71de51a9b08f2e50cde86a944f724624233"} Sep 09 21:03:22.093373 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:22.093350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" event={"ID":"e89d8e1d-d6fc-4236-8b4b-de98a07b1555","Type":"ContainerStarted","Data":"70dfc0e31eaed28b7d36cd58f1bf7728c6e8f99b1e7f34daed77e64e2a6baebf"} Sep 09 21:03:22.094536 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:22.094514 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" event={"ID":"2cff4cd5-a2b8-4871-8f70-37a239f688ce","Type":"ContainerStarted","Data":"b35f4edccad8b014ec7cd1e5971dcb658af39c429c7ea220edffc8b8dbbf8a7f"} Sep 09 21:03:22.117593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:22.117554 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-787f6d466-dsx87" podStartSLOduration=1.117543202 podStartE2EDuration="1.117543202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:22.117294825 +0000 UTC m=+104.931144975" watchObservedRunningTime="2026-09-09 21:03:22.117543202 +0000 UTC m=+104.931393342" Sep 09 21:03:24.104822 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:24.104778 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" event={"ID":"e89d8e1d-d6fc-4236-8b4b-de98a07b1555","Type":"ContainerStarted","Data":"9325446f3591cde91153bf3914b073be26da66b45afcb0ac61c83ff1b24b64a6"} Sep 09 21:03:24.105301 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:24.104979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" Sep 09 21:03:24.109700 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:24.109675 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" Sep 09 21:03:24.129016 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:24.128968 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-6fp9f" podStartSLOduration=1.650545887 podStartE2EDuration="3.128953627s" totalImagesPullingTime="1.47840774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:24.127943872 +0000 UTC m=+106.941794022" watchObservedRunningTime="2026-09-09 21:03:24.128953627 +0000 UTC m=+106.942803776" Sep 09 21:03:25.110939 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:25.110857 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" event={"ID":"2cff4cd5-a2b8-4871-8f70-37a239f688ce","Type":"ContainerStarted","Data":"636e225bd00cabd61663533d928a3a40b5c282d430038ac6536b0c6dfeb1be00"} Sep 09 21:03:25.135926 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:25.135882 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-574d9f77bd-nngl6" podStartSLOduration=1.148535176 podStartE2EDuration="4.135866413s" totalImagesPullingTime="2.987331237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:25.135048938 +0000 UTC m=+107.948899087" watchObservedRunningTime="2026-09-09 21:03:25.135866413 +0000 UTC m=+107.949716562" Sep 09 21:03:30.349980 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.349951 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-z8p25"] Sep 09 21:03:30.353074 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.353057 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.356337 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.356318 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 09 21:03:30.356478 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.356407 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 09 21:03:30.356478 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.356463 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fpddh\"" Sep 09 21:03:30.356615 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.356546 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 21:03:30.356615 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.356570 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 21:03:30.356714 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.356695 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 09 21:03:30.356778 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.356764 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 21:03:30.361290 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.361265 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pg6zh"] Sep 09 21:03:30.363246 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.363230 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.365628 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.365607 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 21:03:30.365699 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.365632 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-brllz\"" Sep 09 21:03:30.365699 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.365647 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 21:03:30.365855 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.365843 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 21:03:30.368678 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.368661 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-z8p25"] Sep 09 21:03:30.498486 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9178b7a9-1800-443a-aa84-9b8d54687c22-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.498486 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.498701 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7mzz\" (UniqueName: \"kubernetes.io/projected/f456c1db-4b9b-4334-82e1-9317eeb17180-kube-api-access-b7mzz\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498701 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.498701 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-sys\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498701 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-textfile\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498701 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498628 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9178b7a9-1800-443a-aa84-9b8d54687c22-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.498701 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498650 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-tls\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498701 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-accelerators-collector-config\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498927 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f456c1db-4b9b-4334-82e1-9317eeb17180-metrics-client-ca\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498927 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498927 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-wtmp\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.498927 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.498927 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498789 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmlh\" (UniqueName: \"kubernetes.io/projected/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-api-access-9zmlh\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.498927 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.498803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-root\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.599679 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f456c1db-4b9b-4334-82e1-9317eeb17180-metrics-client-ca\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.599838 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.599838 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-wtmp\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.599838 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.599977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmlh\" (UniqueName: \"kubernetes.io/projected/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-api-access-9zmlh\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.599977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599875 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-wtmp\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.599977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-root\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.599977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599938 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-root\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.599977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.599943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9178b7a9-1800-443a-aa84-9b8d54687c22-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.600239 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.600239 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7mzz\" (UniqueName: \"kubernetes.io/projected/f456c1db-4b9b-4334-82e1-9317eeb17180-kube-api-access-b7mzz\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.600239 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600159 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.600239 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600234 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-sys\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.600453 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-textfile\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.600453 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9178b7a9-1800-443a-aa84-9b8d54687c22-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.600453 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-tls\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.600453 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600358 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f456c1db-4b9b-4334-82e1-9317eeb17180-metrics-client-ca\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.600722 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-accelerators-collector-config\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.600808 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:03:30.600270 2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 09 21:03:30.600861 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600814 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9178b7a9-1800-443a-aa84-9b8d54687c22-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.600861 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:03:30.600845 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-tls podName:9178b7a9-1800-443a-aa84-9b8d54687c22 nodeName:}" failed. No retries permitted until 2026-09-09 21:03:31.100824976 +0000 UTC m=+113.914675132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-z8p25" (UID: "9178b7a9-1800-443a-aa84-9b8d54687c22") : secret "kube-state-metrics-tls" not found Sep 09 21:03:30.601308 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.600961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.601308 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.601029 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-accelerators-collector-config\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.601308 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.601042 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f456c1db-4b9b-4334-82e1-9317eeb17180-sys\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.601308 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.601102 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-textfile\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.601541 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.601457 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9178b7a9-1800-443a-aa84-9b8d54687c22-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.602514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.602494 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.602769 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.602749 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.603306 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.603287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f456c1db-4b9b-4334-82e1-9317eeb17180-node-exporter-tls\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.612545 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.612520 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmlh\" (UniqueName: \"kubernetes.io/projected/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-api-access-9zmlh\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:30.613103 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.613082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7mzz\" (UniqueName: \"kubernetes.io/projected/f456c1db-4b9b-4334-82e1-9317eeb17180-kube-api-access-b7mzz\") pod \"node-exporter-pg6zh\" (UID: \"f456c1db-4b9b-4334-82e1-9317eeb17180\") " pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.672270 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:30.672250 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pg6zh" Sep 09 21:03:30.682415 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:30.682385 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf456c1db_4b9b_4334_82e1_9317eeb17180.slice/crio-e482047f4994852599b28ec6884eeab909a3e70cd2bebf5de9c9df903fd4ffac WatchSource:0}: Error finding container e482047f4994852599b28ec6884eeab909a3e70cd2bebf5de9c9df903fd4ffac: Status 404 returned error can't find the container with id e482047f4994852599b28ec6884eeab909a3e70cd2bebf5de9c9df903fd4ffac Sep 09 21:03:31.097136 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:31.097106 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zs7k6" Sep 09 21:03:31.104264 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:31.104240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:31.106686 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:31.106668 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9178b7a9-1800-443a-aa84-9b8d54687c22-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-z8p25\" (UID: \"9178b7a9-1800-443a-aa84-9b8d54687c22\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:31.131628 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:31.131600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pg6zh" event={"ID":"f456c1db-4b9b-4334-82e1-9317eeb17180","Type":"ContainerStarted","Data":"e482047f4994852599b28ec6884eeab909a3e70cd2bebf5de9c9df903fd4ffac"} Sep 09 21:03:31.262728 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:31.262697 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" Sep 09 21:03:31.312869 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:31.312263 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:03:31.543437 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:31.543271 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-z8p25"] Sep 09 21:03:31.642296 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:31.642253 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9178b7a9_1800_443a_aa84_9b8d54687c22.slice/crio-b8632128ca5ab08af17b3c3e656d1326a67aabd73c9f393deae4a386d566374d WatchSource:0}: Error finding container b8632128ca5ab08af17b3c3e656d1326a67aabd73c9f393deae4a386d566374d: Status 404 returned error can't find the container with id b8632128ca5ab08af17b3c3e656d1326a67aabd73c9f393deae4a386d566374d Sep 09 21:03:32.136535 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:32.136449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" event={"ID":"9178b7a9-1800-443a-aa84-9b8d54687c22","Type":"ContainerStarted","Data":"b8632128ca5ab08af17b3c3e656d1326a67aabd73c9f393deae4a386d566374d"} Sep 09 21:03:32.138006 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:32.137979 2578 generic.go:356] "Generic (PLEG): container finished" podID="f456c1db-4b9b-4334-82e1-9317eeb17180" containerID="18ef62678f539a6f5fd710905500fda10d8cbeaf1614cedb4b1b3035eca4a0f7" exitCode=0 Sep 09 21:03:32.138113 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:32.138014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pg6zh" event={"ID":"f456c1db-4b9b-4334-82e1-9317eeb17180","Type":"ContainerDied","Data":"18ef62678f539a6f5fd710905500fda10d8cbeaf1614cedb4b1b3035eca4a0f7"} Sep 09 21:03:33.143998 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:33.143963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pg6zh" event={"ID":"f456c1db-4b9b-4334-82e1-9317eeb17180","Type":"ContainerStarted","Data":"5885920f036cb7eb2b6ec7a7605fed190d595b0795a3f22cb9abf065ea1837ee"} Sep 09 21:03:33.144461 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:33.144022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pg6zh" event={"ID":"f456c1db-4b9b-4334-82e1-9317eeb17180","Type":"ContainerStarted","Data":"5ab513ca515a5a4d01331916052d31bc8cf97710f0df277285e24badda3843be"} Sep 09 21:03:33.189267 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:33.189187 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pg6zh" podStartSLOduration=2.138293995 podStartE2EDuration="3.189165152s" totalImagesPullingTime="983.243579ms" totalInitContainerRuntime="67.627578ms" isStatefulPod=true podCreationTimestamp="2026-09-09 21:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:33.186653851 +0000 UTC m=+116.000504013" watchObservedRunningTime="2026-09-09 21:03:33.189165152 +0000 UTC m=+116.003015302" Sep 09 21:03:34.149211 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:34.149172 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" event={"ID":"9178b7a9-1800-443a-aa84-9b8d54687c22","Type":"ContainerStarted","Data":"eabb9dabb68335c60c8fbcca612c26098fbd68176f6901b4f2b142bb37fb1b27"} Sep 09 21:03:34.149566 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:34.149241 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" event={"ID":"9178b7a9-1800-443a-aa84-9b8d54687c22","Type":"ContainerStarted","Data":"977d87755ad581053082ea2dc14fdfaa1d2bdae64da73d17bde487263a08aefa"} Sep 09 21:03:34.149566 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:34.149268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" event={"ID":"9178b7a9-1800-443a-aa84-9b8d54687c22","Type":"ContainerStarted","Data":"bb3decfb81bdfca5f59244feb618e812bab9c89d7d5e01fc9dceff8a7927e39b"} Sep 09 21:03:34.175744 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:34.175697 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-z8p25" podStartSLOduration=2.4909017540000002 podStartE2EDuration="4.17568347s" totalImagesPullingTime="1.684781716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:34.174441839 +0000 UTC m=+116.988292000" watchObservedRunningTime="2026-09-09 21:03:34.17568347 +0000 UTC m=+116.989533619" Sep 09 21:03:35.152150 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.152116 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg"] Sep 09 21:03:35.154576 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.154556 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.158268 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.158250 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 09 21:03:35.158336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.158250 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jhdhr\"" Sep 09 21:03:35.170867 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.170844 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg"] Sep 09 21:03:35.240518 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.240489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6cf17b4a-d499-43e7-9595-87b891bbab5a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pbsmg\" (UID: \"6cf17b4a-d499-43e7-9595-87b891bbab5a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.240677 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.240547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npf8c\" (UniqueName: \"kubernetes.io/projected/6cf17b4a-d499-43e7-9595-87b891bbab5a-kube-api-access-npf8c\") pod \"monitoring-plugin-85bbbbbf4-pbsmg\" (UID: \"6cf17b4a-d499-43e7-9595-87b891bbab5a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.341075 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.341047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6cf17b4a-d499-43e7-9595-87b891bbab5a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pbsmg\" (UID: \"6cf17b4a-d499-43e7-9595-87b891bbab5a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.341160 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.341099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npf8c\" (UniqueName: \"kubernetes.io/projected/6cf17b4a-d499-43e7-9595-87b891bbab5a-kube-api-access-npf8c\") pod \"monitoring-plugin-85bbbbbf4-pbsmg\" (UID: \"6cf17b4a-d499-43e7-9595-87b891bbab5a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.343550 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.343533 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6cf17b4a-d499-43e7-9595-87b891bbab5a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pbsmg\" (UID: \"6cf17b4a-d499-43e7-9595-87b891bbab5a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.353126 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.353104 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npf8c\" (UniqueName: \"kubernetes.io/projected/6cf17b4a-d499-43e7-9595-87b891bbab5a-kube-api-access-npf8c\") pod \"monitoring-plugin-85bbbbbf4-pbsmg\" (UID: \"6cf17b4a-d499-43e7-9595-87b891bbab5a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.463450 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.463370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:35.624338 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.624305 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-785db4f4fc-kzj5f"] Sep 09 21:03:35.627291 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.627274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.629775 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.629747 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 09 21:03:35.629886 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.629793 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 09 21:03:35.630049 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.630034 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 09 21:03:35.630177 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.630160 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 09 21:03:35.630307 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.630288 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 09 21:03:35.630429 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.630411 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kt5kj\"" Sep 09 21:03:35.635189 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.635169 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 09 21:03:35.642708 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.642690 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-785db4f4fc-kzj5f"] Sep 09 21:03:35.643538 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-secret-telemeter-client\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.643597 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-telemeter-trusted-ca-bundle\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.643641 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-serving-certs-ca-bundle\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.643677 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-federate-client-tls\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.643728 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643676 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-telemeter-client-tls\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.643774 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.643826 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drsn\" (UniqueName: \"kubernetes.io/projected/12327d20-457a-493c-9073-639b020da874-kube-api-access-8drsn\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.643877 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.643846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-metrics-client-ca\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.732117 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.731997 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg"] Sep 09 21:03:35.734796 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:35.734769 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf17b4a_d499_43e7_9595_87b891bbab5a.slice/crio-f66e7f0dab49c57d73db3c89b55a621b6e3c3e17f8ca1abfccf5862c65a3f293 WatchSource:0}: Error finding container f66e7f0dab49c57d73db3c89b55a621b6e3c3e17f8ca1abfccf5862c65a3f293: Status 404 returned error can't find the container with id f66e7f0dab49c57d73db3c89b55a621b6e3c3e17f8ca1abfccf5862c65a3f293 Sep 09 21:03:35.744265 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-secret-telemeter-client\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.744361 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-telemeter-trusted-ca-bundle\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.744361 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-serving-certs-ca-bundle\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.744465 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-federate-client-tls\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.744465 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-telemeter-client-tls\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.744465 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.744603 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drsn\" (UniqueName: \"kubernetes.io/projected/12327d20-457a-493c-9073-639b020da874-kube-api-access-8drsn\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.744603 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.744522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-metrics-client-ca\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.745063 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.745042 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-serving-certs-ca-bundle\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.745542 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.745517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-metrics-client-ca\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.745684 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.745661 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12327d20-457a-493c-9073-639b020da874-telemeter-trusted-ca-bundle\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.746896 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.746875 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-secret-telemeter-client\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.746999 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.746975 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-federate-client-tls\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.747422 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.747399 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.747474 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.747463 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12327d20-457a-493c-9073-639b020da874-telemeter-client-tls\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.756569 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.756553 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drsn\" (UniqueName: \"kubernetes.io/projected/12327d20-457a-493c-9073-639b020da874-kube-api-access-8drsn\") pod \"telemeter-client-785db4f4fc-kzj5f\" (UID: \"12327d20-457a-493c-9073-639b020da874\") " pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:35.937798 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:35.937766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" Sep 09 21:03:36.157091 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:36.157053 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" event={"ID":"6cf17b4a-d499-43e7-9595-87b891bbab5a","Type":"ContainerStarted","Data":"f66e7f0dab49c57d73db3c89b55a621b6e3c3e17f8ca1abfccf5862c65a3f293"} Sep 09 21:03:36.219620 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:36.219598 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-785db4f4fc-kzj5f"] Sep 09 21:03:36.221121 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:36.221091 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12327d20_457a_493c_9073_639b020da874.slice/crio-d7ec0a6f170b0d4b576d2493879e9cbc32c634ecfe93561b18d4e989e72cc7f4 WatchSource:0}: Error finding container d7ec0a6f170b0d4b576d2493879e9cbc32c634ecfe93561b18d4e989e72cc7f4: Status 404 returned error can't find the container with id d7ec0a6f170b0d4b576d2493879e9cbc32c634ecfe93561b18d4e989e72cc7f4 Sep 09 21:03:37.163083 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:37.163043 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" event={"ID":"12327d20-457a-493c-9073-639b020da874","Type":"ContainerStarted","Data":"d7ec0a6f170b0d4b576d2493879e9cbc32c634ecfe93561b18d4e989e72cc7f4"} Sep 09 21:03:38.169805 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:38.169738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" event={"ID":"6cf17b4a-d499-43e7-9595-87b891bbab5a","Type":"ContainerStarted","Data":"873ec136cce89118cd65e150f861da1469d5beb6fd2ac231412c0b99e700d482"} Sep 09 21:03:38.170288 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:38.170118 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:38.176377 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:38.176356 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" Sep 09 21:03:38.195533 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:38.195491 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pbsmg" podStartSLOduration=1.436692453 podStartE2EDuration="3.195475012s" totalImagesPullingTime="1.758782559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:38.194560311 +0000 UTC m=+121.008410451" watchObservedRunningTime="2026-09-09 21:03:38.195475012 +0000 UTC m=+121.009325163" Sep 09 21:03:39.175467 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:39.175423 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" event={"ID":"12327d20-457a-493c-9073-639b020da874","Type":"ContainerStarted","Data":"b411345f415785bab3096d531a1af1c06ecd7e31ae2b3eb6fe6c6aba457aae94"} Sep 09 21:03:40.182360 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:40.182326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" event={"ID":"12327d20-457a-493c-9073-639b020da874","Type":"ContainerStarted","Data":"1dd1164b3b42912ab62eb869346612630eac9e83cb7a074fde2d256eee2acd5d"} Sep 09 21:03:40.182802 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:40.182384 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" event={"ID":"12327d20-457a-493c-9073-639b020da874","Type":"ContainerStarted","Data":"df81075f4468606c4655ea1c54aa8bfdcbe992cbbf7106b27ef7c8158c71c84d"} Sep 09 21:03:40.216856 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:40.216797 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-785db4f4fc-kzj5f" podStartSLOduration=2.050290195 podStartE2EDuration="5.216779511s" totalImagesPullingTime="3.166489316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:03:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:40.214762085 +0000 UTC m=+123.028612257" watchObservedRunningTime="2026-09-09 21:03:40.216779511 +0000 UTC m=+123.030629663" Sep 09 21:03:43.104988 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:43.104961 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-787f6d466-dsx87" Sep 09 21:03:46.325723 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.325682 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" podUID="85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" containerName="registry" containerID="cri-o://f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827" gracePeriod=30 Sep 09 21:03:46.563691 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.563669 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:03:46.634271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634178 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-bound-sa-token\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634247 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-ca-trust-extracted\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634271 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634270 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-trusted-ca\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634519 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634298 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-image-registry-private-configuration\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634519 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634317 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-installation-pull-secrets\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634519 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634356 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvk77\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-kube-api-access-jvk77\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634519 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634401 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-certificates\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634519 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634433 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") pod \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\" (UID: \"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e\") " Sep 09 21:03:46.634761 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634734 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:03:46.634828 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634804 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:03:46.634891 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634875 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-trusted-ca\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:46.634930 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.634901 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-certificates\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:46.636850 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.636822 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:03:46.636985 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.636961 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-kube-api-access-jvk77" (OuterVolumeSpecName: "kube-api-access-jvk77") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "kube-api-access-jvk77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:03:46.637103 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.637086 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:03:46.637434 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.637408 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:03:46.637538 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.637515 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:03:46.643365 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.643344 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" (UID: "85d72fdd-c47c-4c25-9b74-26ce40bb0f6e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:03:46.735540 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.735512 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-registry-tls\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:46.735540 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.735535 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-bound-sa-token\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:46.735540 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.735544 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-ca-trust-extracted\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:46.735716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.735553 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-image-registry-private-configuration\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:46.735716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.735563 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-installation-pull-secrets\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:46.735716 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:46.735572 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvk77\" (UniqueName: \"kubernetes.io/projected/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e-kube-api-access-jvk77\") on node \"ip-10-0-133-18.ec2.internal\" DevicePath \"\"" Sep 09 21:03:47.214283 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.214242 2578 generic.go:356] "Generic (PLEG): container finished" podID="85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" containerID="f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827" exitCode=0 Sep 09 21:03:47.214472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.214287 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" Sep 09 21:03:47.214472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.214295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" event={"ID":"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e","Type":"ContainerDied","Data":"f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827"} Sep 09 21:03:47.214472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.214344 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-647c9c8b45-hd7sg" event={"ID":"85d72fdd-c47c-4c25-9b74-26ce40bb0f6e","Type":"ContainerDied","Data":"9edf63ba960176021e2514b5ebe9099d49ce3cdb5745dcd4a120dface24de428"} Sep 09 21:03:47.214472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.214369 2578 scope.go:117] "RemoveContainer" containerID="f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827" Sep 09 21:03:47.222974 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.222958 2578 scope.go:117] "RemoveContainer" containerID="f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827" Sep 09 21:03:47.223274 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:03:47.223254 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827\": container with ID starting with f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827 not found: ID does not exist" containerID="f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827" Sep 09 21:03:47.223321 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.223285 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827"} err="failed to get container status \"f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827\": rpc error: code = NotFound desc = could not find container \"f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827\": container with ID starting with f52f3432b19df9001ac64fabab15638d65e2985ba8269d9a7a8af1d20d7ee827 not found: ID does not exist" Sep 09 21:03:47.245983 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.245955 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-647c9c8b45-hd7sg"] Sep 09 21:03:47.257840 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.257817 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-647c9c8b45-hd7sg"] Sep 09 21:03:47.440067 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.440028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:03:47.442495 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.442476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e6ddbb9-c28f-44da-b932-5f722aab46f4-metrics-certs\") pod \"network-metrics-daemon-d9g6l\" (UID: \"5e6ddbb9-c28f-44da-b932-5f722aab46f4\") " pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:03:47.685677 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.685644 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" path="/var/lib/kubelet/pods/85d72fdd-c47c-4c25-9b74-26ce40bb0f6e/volumes" Sep 09 21:03:47.708514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.708496 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvxk\"" Sep 09 21:03:47.716509 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:47.716491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-d9g6l" Sep 09 21:03:48.004222 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:48.004169 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-d9g6l"] Sep 09 21:03:48.007094 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:03:48.007071 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6ddbb9_c28f_44da_b932_5f722aab46f4.slice/crio-978d0e6aec1d8e635c7fe4ca27d93dba29c7c26d05e08b9ecb84967cb8a91cdb WatchSource:0}: Error finding container 978d0e6aec1d8e635c7fe4ca27d93dba29c7c26d05e08b9ecb84967cb8a91cdb: Status 404 returned error can't find the container with id 978d0e6aec1d8e635c7fe4ca27d93dba29c7c26d05e08b9ecb84967cb8a91cdb Sep 09 21:03:48.220183 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:48.220151 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d9g6l" event={"ID":"5e6ddbb9-c28f-44da-b932-5f722aab46f4","Type":"ContainerStarted","Data":"978d0e6aec1d8e635c7fe4ca27d93dba29c7c26d05e08b9ecb84967cb8a91cdb"} Sep 09 21:03:49.225674 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:49.225639 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d9g6l" event={"ID":"5e6ddbb9-c28f-44da-b932-5f722aab46f4","Type":"ContainerStarted","Data":"542ee827874c2877b52731245e0ec6140b1f9e4d13f14c26b72dc84e3ef04f78"} Sep 09 21:03:49.226073 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:49.225698 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-d9g6l" event={"ID":"5e6ddbb9-c28f-44da-b932-5f722aab46f4","Type":"ContainerStarted","Data":"7bbfa45177373597c7f573cb3067e0c5a1e8bb27909e4fae37702caf57cbdba4"} Sep 09 21:03:49.250242 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:03:49.250172 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-d9g6l" podStartSLOduration=130.298639453 podStartE2EDuration="2m11.250157863s" totalImagesPullingTime="951.51841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:03:49.249630231 +0000 UTC m=+132.063480380" watchObservedRunningTime="2026-09-09 21:03:49.250157863 +0000 UTC m=+132.064008012" Sep 09 21:04:05.163578 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:04:05.163554 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lld6z_6951edbb-68bf-4b11-8bbc-33b960a5c4d5/serve-healthcheck-canary/0.log" Sep 09 21:06:37.566561 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:06:37.566528 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:06:37.569120 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:06:37.569091 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:06:37.577903 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:06:37.577885 2578 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 21:08:34.677849 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.677822 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 21:08:34.678348 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.678090 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" containerName="registry" Sep 09 21:08:34.678348 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.678100 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" containerName="registry" Sep 09 21:08:34.678348 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.678149 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="85d72fdd-c47c-4c25-9b74-26ce40bb0f6e" containerName="registry" Sep 09 21:08:34.681507 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.681488 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.684670 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.684650 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 09 21:08:34.685070 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685054 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 09 21:08:34.685172 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685155 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 09 21:08:34.685428 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685411 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 09 21:08:34.685498 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685427 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mprzv\"" Sep 09 21:08:34.685555 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685502 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 09 21:08:34.685555 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685486 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 09 21:08:34.685654 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685640 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 09 21:08:34.685706 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.685644 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 09 21:08:34.691784 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.691766 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 09 21:08:34.713950 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.713922 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 21:08:34.777758 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/735f6f0f-a247-4302-bb56-7c4de81a2176-tls-assets\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.777872 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/735f6f0f-a247-4302-bb56-7c4de81a2176-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.777929 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.777929 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777901 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.777929 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777916 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-web-config\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777933 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/735f6f0f-a247-4302-bb56-7c4de81a2176-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735f6f0f-a247-4302-bb56-7c4de81a2176-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.777975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-config-volume\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.778022 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.778056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778386 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.778091 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/735f6f0f-a247-4302-bb56-7c4de81a2176-config-out\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778386 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.778115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2lkc\" (UniqueName: \"kubernetes.io/projected/735f6f0f-a247-4302-bb56-7c4de81a2176-kube-api-access-z2lkc\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.778386 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.778146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878444 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/735f6f0f-a247-4302-bb56-7c4de81a2176-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-web-config\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/735f6f0f-a247-4302-bb56-7c4de81a2176-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735f6f0f-a247-4302-bb56-7c4de81a2176-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878583 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-config-volume\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/735f6f0f-a247-4302-bb56-7c4de81a2176-config-out\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2lkc\" (UniqueName: \"kubernetes.io/projected/735f6f0f-a247-4302-bb56-7c4de81a2176-kube-api-access-z2lkc\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878742 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/735f6f0f-a247-4302-bb56-7c4de81a2176-tls-assets\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.878898 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.878840 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/735f6f0f-a247-4302-bb56-7c4de81a2176-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.879312 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.879291 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/735f6f0f-a247-4302-bb56-7c4de81a2176-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.880336 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.879702 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/735f6f0f-a247-4302-bb56-7c4de81a2176-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.881928 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.881786 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/735f6f0f-a247-4302-bb56-7c4de81a2176-tls-assets\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.881928 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.881901 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.882521 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.882314 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-web-config\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.882521 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.882418 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-config-volume\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.882521 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.882463 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.882521 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.882467 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.882687 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.882621 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/735f6f0f-a247-4302-bb56-7c4de81a2176-config-out\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.882868 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.882847 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.883896 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.883880 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/735f6f0f-a247-4302-bb56-7c4de81a2176-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.888438 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.888417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2lkc\" (UniqueName: \"kubernetes.io/projected/735f6f0f-a247-4302-bb56-7c4de81a2176-kube-api-access-z2lkc\") pod \"alertmanager-main-0\" (UID: \"735f6f0f-a247-4302-bb56-7c4de81a2176\") " pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:34.990293 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:34.990271 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 09 21:08:35.288407 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:35.288379 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 09 21:08:35.289614 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:08:35.289591 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod735f6f0f_a247_4302_bb56_7c4de81a2176.slice/crio-6adff85b1bb17cf3b7fff4b303b3183c971694dd5bf8b33d26151e1bcc559d5d WatchSource:0}: Error finding container 6adff85b1bb17cf3b7fff4b303b3183c971694dd5bf8b33d26151e1bcc559d5d: Status 404 returned error can't find the container with id 6adff85b1bb17cf3b7fff4b303b3183c971694dd5bf8b33d26151e1bcc559d5d Sep 09 21:08:36.244964 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:36.240636 2578 generic.go:356] "Generic (PLEG): container finished" podID="735f6f0f-a247-4302-bb56-7c4de81a2176" containerID="aef9e2c403c1ab50805827fd93c7810931e8c356e2bf6db72ca5a777d1655793" exitCode=0 Sep 09 21:08:36.244964 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:36.240687 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerDied","Data":"aef9e2c403c1ab50805827fd93c7810931e8c356e2bf6db72ca5a777d1655793"} Sep 09 21:08:36.244964 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:36.240819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerStarted","Data":"6adff85b1bb17cf3b7fff4b303b3183c971694dd5bf8b33d26151e1bcc559d5d"} Sep 09 21:08:36.245914 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:36.245895 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:08:38.252135 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:38.252099 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerStarted","Data":"04d691acc33852b428f15aa044a449905bd939543e784a17913d2193ed0f4e32"} Sep 09 21:08:38.252512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:38.252173 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerStarted","Data":"49a51518167ba2b4694fff9a974cddc0944944b9f0238b04d135877c4913e64c"} Sep 09 21:08:38.252512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:38.252233 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerStarted","Data":"68680c82e4568211a7d58eb316640d3c4af274b5b8178bc9c16a464c783c6053"} Sep 09 21:08:38.252512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:38.252260 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerStarted","Data":"827d6688f4e0c42a5f43584842f39cf0bd552fa57d6dabeab78f29de4ebcfb40"} Sep 09 21:08:38.252512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:38.252290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerStarted","Data":"ca730bd818a9e71eba92911581521177c0c88ed8a3d2e3e99633f29f7a3b640d"} Sep 09 21:08:40.016977 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.016930 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 21:08:40.020734 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.020718 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.027116 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.027054 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 21:08:40.027833 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.027806 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 21:08:40.027941 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.027869 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 21:08:40.027941 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.027894 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 21:08:40.028365 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028345 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 21:08:40.028482 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028386 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 21:08:40.028482 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028394 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 21:08:40.028482 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028446 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bkvd1709qap7r\"" Sep 09 21:08:40.028629 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028397 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sllfl\"" Sep 09 21:08:40.028629 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028398 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 21:08:40.028749 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028736 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 21:08:40.028939 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.028924 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 21:08:40.029451 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.029428 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 21:08:40.031743 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.031727 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 21:08:40.036828 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.036814 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 21:08:40.037272 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.037257 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 21:08:40.038850 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.038836 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 21:08:40.045548 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.045525 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 21:08:40.119484 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9ccr\" (UniqueName: \"kubernetes.io/projected/aa50c80b-9513-498f-b7d9-3a0216a246d4-kube-api-access-t9ccr\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119516 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119593 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119555 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa50c80b-9513-498f-b7d9-3a0216a246d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa50c80b-9513-498f-b7d9-3a0216a246d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119866 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119742 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119866 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119866 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119866 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119866 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119843 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-config\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.119866 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.120036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.120036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.120036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.120036 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.119991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.220865 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.220842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221014 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.220876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221014 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.220891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221014 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.220912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221014 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.220937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221014 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.220961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9ccr\" (UniqueName: \"kubernetes.io/projected/aa50c80b-9513-498f-b7d9-3a0216a246d4-kube-api-access-t9ccr\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221014 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.220986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221014 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa50c80b-9513-498f-b7d9-3a0216a246d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa50c80b-9513-498f-b7d9-3a0216a246d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221362 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-config\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.221757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.221454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.223795 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.222025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.223795 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.222339 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.223795 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.223070 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.223795 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.223137 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.224066 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.224004 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.224518 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.224343 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.225039 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.224987 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.225130 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.225045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa50c80b-9513-498f-b7d9-3a0216a246d4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.225383 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.225345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.225488 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.225438 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.226012 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.225564 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.226012 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.225970 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.226012 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.225972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.226616 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.226589 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.226890 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.226866 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.227104 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.227082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-config\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.227275 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.227258 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa50c80b-9513-498f-b7d9-3a0216a246d4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.227647 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.227628 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa50c80b-9513-498f-b7d9-3a0216a246d4-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.227708 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.227676 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa50c80b-9513-498f-b7d9-3a0216a246d4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.232733 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.232714 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9ccr\" (UniqueName: \"kubernetes.io/projected/aa50c80b-9513-498f-b7d9-3a0216a246d4-kube-api-access-t9ccr\") pod \"prometheus-k8s-0\" (UID: \"aa50c80b-9513-498f-b7d9-3a0216a246d4\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.262817 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.262790 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"735f6f0f-a247-4302-bb56-7c4de81a2176","Type":"ContainerStarted","Data":"b631dce4372f33ceeb136065f611806485b6179e503af00fb3af85c02103ff43"} Sep 09 21:08:40.298374 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.298296 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.400601078 podStartE2EDuration="6.2982825s" totalImagesPullingTime="2.883137223s" totalInitContainerRuntime="14.544199ms" isStatefulPod=false podCreationTimestamp="2026-09-09 21:08:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:08:40.295536951 +0000 UTC m=+423.109387099" watchObservedRunningTime="2026-09-09 21:08:40.2982825 +0000 UTC m=+423.112132649" Sep 09 21:08:40.330788 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.330758 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:08:40.629564 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:40.629494 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 21:08:40.632249 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:08:40.632221 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa50c80b_9513_498f_b7d9_3a0216a246d4.slice/crio-8f98f4069635b76350078b654715a9bedbb505a2bbdf03a303106b136c268259 WatchSource:0}: Error finding container 8f98f4069635b76350078b654715a9bedbb505a2bbdf03a303106b136c268259: Status 404 returned error can't find the container with id 8f98f4069635b76350078b654715a9bedbb505a2bbdf03a303106b136c268259 Sep 09 21:08:41.269048 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:41.269019 2578 generic.go:356] "Generic (PLEG): container finished" podID="aa50c80b-9513-498f-b7d9-3a0216a246d4" containerID="85a3b4991a99a037e3c11441fd79f30db0f72fc2ceb43d98fe99b76f81741cc8" exitCode=0 Sep 09 21:08:41.269528 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:41.269109 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerDied","Data":"85a3b4991a99a037e3c11441fd79f30db0f72fc2ceb43d98fe99b76f81741cc8"} Sep 09 21:08:41.269528 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:41.269218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerStarted","Data":"8f98f4069635b76350078b654715a9bedbb505a2bbdf03a303106b136c268259"} Sep 09 21:08:45.292166 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:45.292121 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerStarted","Data":"11a2c49f5b8c553859b18619d378ddbaac1846502c059461d863aaac939a704b"} Sep 09 21:08:45.292609 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:45.292226 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerStarted","Data":"a6cb6aa4d494121172c42b08f68635b918ca3f3b7bc35d3b57ecb5bcf03e70cc"} Sep 09 21:08:47.311176 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:47.311139 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerStarted","Data":"e7a30b2b0ab5109dbb7151d9c59d0332625c9f8245de8eac41390ef4ea858ce8"} Sep 09 21:08:47.311758 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:47.311247 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerStarted","Data":"8ae8bf016fbe877bafaf16927fd80acb48a9aa8f0f8959258edf361040d4c74f"} Sep 09 21:08:47.311758 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:47.311273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerStarted","Data":"9c91c5ba607a794e2e40fed96ca370e0389e0547c2230b74118505bd9a9c4787"} Sep 09 21:08:47.311758 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:47.311298 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa50c80b-9513-498f-b7d9-3a0216a246d4","Type":"ContainerStarted","Data":"e2f7f699dfebc21a678cdfda97e5c9e9bb7f42ee9ad78c37722b417195491625"} Sep 09 21:08:47.355118 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:47.355041 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.460442969 podStartE2EDuration="8.355025314s" totalImagesPullingTime="4.878173938s" totalInitContainerRuntime="16.408407ms" isStatefulPod=false podCreationTimestamp="2026-09-09 21:08:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:08:47.352768681 +0000 UTC m=+430.166618829" watchObservedRunningTime="2026-09-09 21:08:47.355025314 +0000 UTC m=+430.168875783" Sep 09 21:08:50.331089 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:08:50.331061 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:09:40.331919 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:09:40.331879 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:09:40.351448 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:09:40.351426 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:09:40.533635 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:09:40.533601 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:11:37.595867 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:11:37.595839 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:11:37.596436 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:11:37.595839 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:12:42.009420 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.009388 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27"] Sep 09 21:12:42.011678 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.011663 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.014454 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.014427 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 21:12:42.014559 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.014504 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 21:12:42.014973 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.014959 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 21:12:42.015068 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.015053 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 21:12:42.015136 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.015084 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c4wl6\"" Sep 09 21:12:42.030523 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.030502 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27"] Sep 09 21:12:42.161812 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.161777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn7b2\" (UniqueName: \"kubernetes.io/projected/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-kube-api-access-nn7b2\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.161969 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.161818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-tmp\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.161969 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.161904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.161969 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.161943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-manifests\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.162087 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.161984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.262851 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.262769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn7b2\" (UniqueName: \"kubernetes.io/projected/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-kube-api-access-nn7b2\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.262851 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.262806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-tmp\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.263024 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.262851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.263024 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.262880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-manifests\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.263024 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.262922 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.263342 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.263317 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-manifests\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.265140 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.265119 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-tmp\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.265467 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.265448 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.265467 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.265456 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.272441 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.272417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn7b2\" (UniqueName: \"kubernetes.io/projected/8a8a4848-38dd-42fb-ae89-9b49e201d5a6-kube-api-access-nn7b2\") pod \"opendatahub-operator-controller-manager-59685ff6b-78d27\" (UID: \"8a8a4848-38dd-42fb-ae89-9b49e201d5a6\") " pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.322247 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.322227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:42.628342 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:42.628242 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27"] Sep 09 21:12:42.631885 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:12:42.631853 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8a4848_38dd_42fb_ae89_9b49e201d5a6.slice/crio-be425862754e451d9e28a484cea150080a6af0d43131605912c9873964ccba66 WatchSource:0}: Error finding container be425862754e451d9e28a484cea150080a6af0d43131605912c9873964ccba66: Status 404 returned error can't find the container with id be425862754e451d9e28a484cea150080a6af0d43131605912c9873964ccba66 Sep 09 21:12:43.214092 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:43.214052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" event={"ID":"8a8a4848-38dd-42fb-ae89-9b49e201d5a6","Type":"ContainerStarted","Data":"be425862754e451d9e28a484cea150080a6af0d43131605912c9873964ccba66"} Sep 09 21:12:46.232370 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:46.232333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" event={"ID":"8a8a4848-38dd-42fb-ae89-9b49e201d5a6","Type":"ContainerStarted","Data":"3a6677a0192d12f2f6b522d2fa840d4bdd1b2be467367f703b6832fa8375c3b0"} Sep 09 21:12:47.238354 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:47.238320 2578 generic.go:356] "Generic (PLEG): container finished" podID="8a8a4848-38dd-42fb-ae89-9b49e201d5a6" containerID="3a6677a0192d12f2f6b522d2fa840d4bdd1b2be467367f703b6832fa8375c3b0" exitCode=0 Sep 09 21:12:47.238757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:47.238371 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" event={"ID":"8a8a4848-38dd-42fb-ae89-9b49e201d5a6","Type":"ContainerDied","Data":"3a6677a0192d12f2f6b522d2fa840d4bdd1b2be467367f703b6832fa8375c3b0"} Sep 09 21:12:48.244311 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:48.244273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" event={"ID":"8a8a4848-38dd-42fb-ae89-9b49e201d5a6","Type":"ContainerStarted","Data":"d51cd948a60687fc8eb0d2279d190715f36a92f724d8168a867a8b57c8d3b989"} Sep 09 21:12:48.244714 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:48.244423 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:12:48.277825 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:48.277785 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" podStartSLOduration=2.700786371 podStartE2EDuration="7.277773177s" totalImagesPullingTime="3.504847759s" totalInitContainerRuntime="1.072139047s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:48.276786763 +0000 UTC m=+671.090636913" watchObservedRunningTime="2026-09-09 21:12:48.277773177 +0000 UTC m=+671.091623324" Sep 09 21:12:59.250503 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:12:59.250472 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59685ff6b-78d27" Sep 09 21:13:04.175478 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.175445 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8"] Sep 09 21:13:04.177844 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.177828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.183933 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.183909 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 09 21:13:04.184072 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.184023 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 09 21:13:04.184072 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.184041 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 09 21:13:04.184072 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.184042 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dfg2x\"" Sep 09 21:13:04.184591 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.184136 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:13:04.184591 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.184427 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 09 21:13:04.199045 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.199018 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8"] Sep 09 21:13:04.248490 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.248456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k656k\" (UniqueName: \"kubernetes.io/projected/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-kube-api-access-k656k\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.248727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.248501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-manager-config\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.248727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.248527 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-metrics-cert\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.248727 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.248595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-cert\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.349123 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.349086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-manager-config\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.349321 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.349140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-metrics-cert\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.349321 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.349168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-cert\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.349321 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.349267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k656k\" (UniqueName: \"kubernetes.io/projected/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-kube-api-access-k656k\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.349860 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.349834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-manager-config\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.351765 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.351743 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-cert\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.351888 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.351864 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-metrics-cert\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.359316 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.359296 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k656k\" (UniqueName: \"kubernetes.io/projected/e39ac3b4-fcb4-4096-bc86-9f563c68d1dc-kube-api-access-k656k\") pod \"lws-controller-manager-57b847cbcb-wkqs8\" (UID: \"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc\") " pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.486991 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.486958 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:04.760136 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:04.760106 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8"] Sep 09 21:13:04.762440 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:13:04.762416 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39ac3b4_fcb4_4096_bc86_9f563c68d1dc.slice/crio-368bce5924545bffbed7b0f5c968631101bf71abb4190f6195d96d88331830a2 WatchSource:0}: Error finding container 368bce5924545bffbed7b0f5c968631101bf71abb4190f6195d96d88331830a2: Status 404 returned error can't find the container with id 368bce5924545bffbed7b0f5c968631101bf71abb4190f6195d96d88331830a2 Sep 09 21:13:05.314974 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:05.314942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" event={"ID":"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc","Type":"ContainerStarted","Data":"368bce5924545bffbed7b0f5c968631101bf71abb4190f6195d96d88331830a2"} Sep 09 21:13:08.331374 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:08.331340 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" event={"ID":"e39ac3b4-fcb4-4096-bc86-9f563c68d1dc","Type":"ContainerStarted","Data":"6af35348735b1ac6d0d4d133226464121a97679dc56370e89b75222489620b94"} Sep 09 21:13:08.331723 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:08.331409 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:08.357469 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:08.357426 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" podStartSLOduration=1.490695924 podStartE2EDuration="4.357410653s" totalImagesPullingTime="2.866714729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:08.356863614 +0000 UTC m=+691.170713764" watchObservedRunningTime="2026-09-09 21:13:08.357410653 +0000 UTC m=+691.171260802" Sep 09 21:13:12.623514 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.623482 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vljx6"] Sep 09 21:13:12.626233 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.626190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:12.630633 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.630614 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8xwwc\"" Sep 09 21:13:12.630905 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.630888 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 09 21:13:12.640138 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.640110 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vljx6"] Sep 09 21:13:12.723713 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.723678 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ce49693-854d-48ca-9682-04be30f5d408-cert\") pod \"odh-model-controller-7998c67dc4-vljx6\" (UID: \"3ce49693-854d-48ca-9682-04be30f5d408\") " pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:12.723868 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.723750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4jt\" (UniqueName: \"kubernetes.io/projected/3ce49693-854d-48ca-9682-04be30f5d408-kube-api-access-jz4jt\") pod \"odh-model-controller-7998c67dc4-vljx6\" (UID: \"3ce49693-854d-48ca-9682-04be30f5d408\") " pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:12.825047 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.825017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ce49693-854d-48ca-9682-04be30f5d408-cert\") pod \"odh-model-controller-7998c67dc4-vljx6\" (UID: \"3ce49693-854d-48ca-9682-04be30f5d408\") " pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:12.825236 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.825063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4jt\" (UniqueName: \"kubernetes.io/projected/3ce49693-854d-48ca-9682-04be30f5d408-kube-api-access-jz4jt\") pod \"odh-model-controller-7998c67dc4-vljx6\" (UID: \"3ce49693-854d-48ca-9682-04be30f5d408\") " pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:12.829463 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.829435 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ce49693-854d-48ca-9682-04be30f5d408-cert\") pod \"odh-model-controller-7998c67dc4-vljx6\" (UID: \"3ce49693-854d-48ca-9682-04be30f5d408\") " pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:12.836558 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.836533 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4jt\" (UniqueName: \"kubernetes.io/projected/3ce49693-854d-48ca-9682-04be30f5d408-kube-api-access-jz4jt\") pod \"odh-model-controller-7998c67dc4-vljx6\" (UID: \"3ce49693-854d-48ca-9682-04be30f5d408\") " pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:12.937333 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:12.937265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:13.217170 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.217146 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hprnw"] Sep 09 21:13:13.219777 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:13:13.219753 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce49693_854d_48ca_9682_04be30f5d408.slice/crio-a471159c48f3f02ec0905f37b999a82c42ba877c25ee7eccf3b15aace6efeb86 WatchSource:0}: Error finding container a471159c48f3f02ec0905f37b999a82c42ba877c25ee7eccf3b15aace6efeb86: Status 404 returned error can't find the container with id a471159c48f3f02ec0905f37b999a82c42ba877c25ee7eccf3b15aace6efeb86 Sep 09 21:13:13.220426 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.220410 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vljx6"] Sep 09 21:13:13.220523 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.220506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.223333 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.223313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 09 21:13:13.223591 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.223575 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2xg8p\"" Sep 09 21:13:13.231600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.231577 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hprnw"] Sep 09 21:13:13.330013 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.329988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bc754e-9e45-42f6-8404-536aa2e3ed6f-tls-certs\") pod \"model-serving-api-54d89bd5fc-hprnw\" (UID: \"c0bc754e-9e45-42f6-8404-536aa2e3ed6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.330172 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.330022 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fb62\" (UniqueName: \"kubernetes.io/projected/c0bc754e-9e45-42f6-8404-536aa2e3ed6f-kube-api-access-4fb62\") pod \"model-serving-api-54d89bd5fc-hprnw\" (UID: \"c0bc754e-9e45-42f6-8404-536aa2e3ed6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.354028 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.353998 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" event={"ID":"3ce49693-854d-48ca-9682-04be30f5d408","Type":"ContainerStarted","Data":"a471159c48f3f02ec0905f37b999a82c42ba877c25ee7eccf3b15aace6efeb86"} Sep 09 21:13:13.430862 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.430831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bc754e-9e45-42f6-8404-536aa2e3ed6f-tls-certs\") pod \"model-serving-api-54d89bd5fc-hprnw\" (UID: \"c0bc754e-9e45-42f6-8404-536aa2e3ed6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.431011 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.430867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fb62\" (UniqueName: \"kubernetes.io/projected/c0bc754e-9e45-42f6-8404-536aa2e3ed6f-kube-api-access-4fb62\") pod \"model-serving-api-54d89bd5fc-hprnw\" (UID: \"c0bc754e-9e45-42f6-8404-536aa2e3ed6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.433428 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.433406 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bc754e-9e45-42f6-8404-536aa2e3ed6f-tls-certs\") pod \"model-serving-api-54d89bd5fc-hprnw\" (UID: \"c0bc754e-9e45-42f6-8404-536aa2e3ed6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.439683 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.439661 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fb62\" (UniqueName: \"kubernetes.io/projected/c0bc754e-9e45-42f6-8404-536aa2e3ed6f-kube-api-access-4fb62\") pod \"model-serving-api-54d89bd5fc-hprnw\" (UID: \"c0bc754e-9e45-42f6-8404-536aa2e3ed6f\") " pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.536359 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.536300 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:13.806972 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:13.806941 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hprnw"] Sep 09 21:13:13.809246 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:13:13.809217 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0bc754e_9e45_42f6_8404_536aa2e3ed6f.slice/crio-1155f1da127d656bc90a7eab36faa708dbb31055ee6765c0caa8163750ec5126 WatchSource:0}: Error finding container 1155f1da127d656bc90a7eab36faa708dbb31055ee6765c0caa8163750ec5126: Status 404 returned error can't find the container with id 1155f1da127d656bc90a7eab36faa708dbb31055ee6765c0caa8163750ec5126 Sep 09 21:13:14.360526 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:14.360471 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" event={"ID":"c0bc754e-9e45-42f6-8404-536aa2e3ed6f","Type":"ContainerStarted","Data":"1155f1da127d656bc90a7eab36faa708dbb31055ee6765c0caa8163750ec5126"} Sep 09 21:13:16.848886 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.848854 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg"] Sep 09 21:13:16.856928 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.856902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.859765 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.859737 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 21:13:16.859932 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.859913 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ssmhd\"" Sep 09 21:13:16.860127 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.860097 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 21:13:16.860215 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.860136 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 21:13:16.877289 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.877266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg"] Sep 09 21:13:16.961163 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j49f\" (UniqueName: \"kubernetes.io/projected/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-kube-api-access-5j49f\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961393 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961173 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961393 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961393 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961329 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961393 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961393 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961381 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961621 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961402 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961621 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:16.961621 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:16.961564 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062245 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062445 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062445 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j49f\" (UniqueName: \"kubernetes.io/projected/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-kube-api-access-5j49f\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062445 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062445 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062657 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062657 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062657 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062657 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.062854 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.062729 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.063148 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.063122 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.063307 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.063275 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.063415 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.063386 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.063476 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.063391 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.065007 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.064988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.065255 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.065224 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.074697 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.074675 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j49f\" (UniqueName: \"kubernetes.io/projected/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-kube-api-access-5j49f\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.078285 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.078267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67b1da2e-49a6-4dd7-8501-f3f45e7fabf1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg\" (UID: \"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.167907 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.167828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:17.377299 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.377150 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" event={"ID":"c0bc754e-9e45-42f6-8404-536aa2e3ed6f","Type":"ContainerStarted","Data":"3f53e58aa535674f399ee63e8cf210e928bb48fe56b614a9ceb0a6eb184d5d01"} Sep 09 21:13:17.377299 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.377255 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:17.378875 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.378852 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" event={"ID":"3ce49693-854d-48ca-9682-04be30f5d408","Type":"ContainerStarted","Data":"7c8558a86e472491eb3cb5cf704aa2c0c5b827a5a4089b02cb174ddf000a4453"} Sep 09 21:13:17.378997 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.378979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:17.408816 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.408774 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" podStartSLOduration=0.948415719 podStartE2EDuration="4.408760319s" totalImagesPullingTime="3.4603446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:17.407045055 +0000 UTC m=+700.220895204" watchObservedRunningTime="2026-09-09 21:13:17.408760319 +0000 UTC m=+700.222610491" Sep 09 21:13:17.433534 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.433447 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" podStartSLOduration=1.393386794 podStartE2EDuration="5.43342983s" totalImagesPullingTime="4.040043036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:17.43161213 +0000 UTC m=+700.245462278" watchObservedRunningTime="2026-09-09 21:13:17.43342983 +0000 UTC m=+700.247279981" Sep 09 21:13:17.528914 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:17.528889 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg"] Sep 09 21:13:17.531254 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:13:17.531225 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b1da2e_49a6_4dd7_8501_f3f45e7fabf1.slice/crio-477cf2576caf7c77b0afeefcce5b342ac4c47320d027c2fbeab1c8ecc97b15fe WatchSource:0}: Error finding container 477cf2576caf7c77b0afeefcce5b342ac4c47320d027c2fbeab1c8ecc97b15fe: Status 404 returned error can't find the container with id 477cf2576caf7c77b0afeefcce5b342ac4c47320d027c2fbeab1c8ecc97b15fe Sep 09 21:13:18.390104 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:18.390060 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" event={"ID":"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1","Type":"ContainerStarted","Data":"477cf2576caf7c77b0afeefcce5b342ac4c47320d027c2fbeab1c8ecc97b15fe"} Sep 09 21:13:19.338125 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:19.338096 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57b847cbcb-wkqs8" Sep 09 21:13:21.114138 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.114094 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:13:21.114472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.114172 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:13:21.114472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.114222 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:13:21.228801 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.228767 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht"] Sep 09 21:13:21.231991 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.231975 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.234966 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.234929 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 21:13:21.234966 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.234929 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 21:13:21.235142 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.234999 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 21:13:21.235401 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.235389 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m9nvg\"" Sep 09 21:13:21.247890 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.247864 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht"] Sep 09 21:13:21.302602 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.302563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59630cf3-f4de-49ba-8f47-37c6922a5e6f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.302751 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.302608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.302751 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.302667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.302751 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.302690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqcrj\" (UniqueName: \"kubernetes.io/projected/59630cf3-f4de-49ba-8f47-37c6922a5e6f-kube-api-access-kqcrj\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.302925 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.302748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.302925 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.302810 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.302925 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.302904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.403574 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.403496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.403574 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.403547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.403774 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.403593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59630cf3-f4de-49ba-8f47-37c6922a5e6f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.403774 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.403617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.403774 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.403660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.403774 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.403690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqcrj\" (UniqueName: \"kubernetes.io/projected/59630cf3-f4de-49ba-8f47-37c6922a5e6f-kube-api-access-kqcrj\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.403774 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.403720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.404638 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.404548 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.406604 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.406577 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.407306 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.407284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59630cf3-f4de-49ba-8f47-37c6922a5e6f-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.407438 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.407356 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.407504 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.407487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.409098 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.409070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" event={"ID":"67b1da2e-49a6-4dd7-8501-f3f45e7fabf1","Type":"ContainerStarted","Data":"2afef443f174c76f26d5ec3f5e5178ce20d6810eb07c7ad94bc785b45c016021"} Sep 09 21:13:21.413703 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.413677 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqcrj\" (UniqueName: \"kubernetes.io/projected/59630cf3-f4de-49ba-8f47-37c6922a5e6f-kube-api-access-kqcrj\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.414101 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.414078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59630cf3-f4de-49ba-8f47-37c6922a5e6f-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-r7tht\" (UID: \"59630cf3-f4de-49ba-8f47-37c6922a5e6f\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.435831 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.435772 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" podStartSLOduration=1.8552138390000001 podStartE2EDuration="5.43575293s" totalImagesPullingTime="3.580539091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:21.433743862 +0000 UTC m=+704.247594011" watchObservedRunningTime="2026-09-09 21:13:21.43575293 +0000 UTC m=+704.249603082" Sep 09 21:13:21.541768 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.541732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:21.857867 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:13:21.857836 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59630cf3_f4de_49ba_8f47_37c6922a5e6f.slice/crio-0633995d31457d0ad9518bb98c7d935eb662fafbbc119781c3f1f6ba8b7bb6cb WatchSource:0}: Error finding container 0633995d31457d0ad9518bb98c7d935eb662fafbbc119781c3f1f6ba8b7bb6cb: Status 404 returned error can't find the container with id 0633995d31457d0ad9518bb98c7d935eb662fafbbc119781c3f1f6ba8b7bb6cb Sep 09 21:13:21.857983 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:21.857957 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht"] Sep 09 21:13:22.168543 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:22.168464 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:22.172743 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:22.172720 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:22.415408 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:22.415370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" event={"ID":"59630cf3-f4de-49ba-8f47-37c6922a5e6f","Type":"ContainerStarted","Data":"0633995d31457d0ad9518bb98c7d935eb662fafbbc119781c3f1f6ba8b7bb6cb"} Sep 09 21:13:22.415779 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:22.415752 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:22.416687 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:22.416667 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg" Sep 09 21:13:24.138875 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:24.138836 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:13:24.139272 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:24.138913 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:13:24.428003 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:24.427908 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" event={"ID":"59630cf3-f4de-49ba-8f47-37c6922a5e6f","Type":"ContainerStarted","Data":"cb7caf7021c61434554fc43615edbe5e693a161963819200042d249e44d63fdb"} Sep 09 21:13:24.428255 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:24.428241 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:13:24.494536 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:24.494484 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" podStartSLOduration=1.2157423 podStartE2EDuration="3.49447022s" totalImagesPullingTime="2.27872792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:24.491963577 +0000 UTC m=+707.305813725" watchObservedRunningTime="2026-09-09 21:13:24.49447022 +0000 UTC m=+707.308320368" Sep 09 21:13:25.434661 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:25.434627 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-r7tht container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 09 21:13:25.435020 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:25.434683 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" podUID="59630cf3-f4de-49ba-8f47-37c6922a5e6f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:28.392757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:28.392650 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-vljx6" Sep 09 21:13:28.394772 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:28.394755 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hprnw" Sep 09 21:13:28.434138 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:13:28.434108 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r7tht" Sep 09 21:15:24.673248 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.673214 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs"] Sep 09 21:15:24.675766 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.675746 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.680233 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.680187 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bjvrf\"" Sep 09 21:15:24.696041 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.696019 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs"] Sep 09 21:15:24.783312 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783478 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783319 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783478 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783478 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783478 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783649 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vwz4\" (UniqueName: \"kubernetes.io/projected/dcdd769d-2f06-4103-bd71-9e53a85d0faf-kube-api-access-5vwz4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783649 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783649 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.783751 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.783657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884331 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884331 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vwz4\" (UniqueName: \"kubernetes.io/projected/dcdd769d-2f06-4103-bd71-9e53a85d0faf-kube-api-access-5vwz4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884562 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884562 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884562 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884718 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884718 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884718 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884718 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884676 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884718 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884921 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884732 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.884921 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884873 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.885042 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.884918 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.885247 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.885220 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.886959 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.886941 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.887224 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.887185 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.894249 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.894225 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dcdd769d-2f06-4103-bd71-9e53a85d0faf-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.894747 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.894727 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vwz4\" (UniqueName: \"kubernetes.io/projected/dcdd769d-2f06-4103-bd71-9e53a85d0faf-kube-api-access-5vwz4\") pod \"maas-default-gateway-openshift-default-78bb78ff78-trpxs\" (UID: \"dcdd769d-2f06-4103-bd71-9e53a85d0faf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:24.987842 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:24.987815 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:25.265219 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.265178 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs"] Sep 09 21:15:25.267232 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:15:25.267187 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcdd769d_2f06_4103_bd71_9e53a85d0faf.slice/crio-34ab9c3db7bf4353b4a26f1ce3c7622d494e7dc9490f6f83d9ba294c366c9000 WatchSource:0}: Error finding container 34ab9c3db7bf4353b4a26f1ce3c7622d494e7dc9490f6f83d9ba294c366c9000: Status 404 returned error can't find the container with id 34ab9c3db7bf4353b4a26f1ce3c7622d494e7dc9490f6f83d9ba294c366c9000 Sep 09 21:15:25.269217 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.269168 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:15:25.269329 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.269259 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:15:25.269329 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.269302 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:15:25.959859 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.959824 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" event={"ID":"dcdd769d-2f06-4103-bd71-9e53a85d0faf","Type":"ContainerStarted","Data":"948a7c72e4d40aa65cfa2dd28e02a86b19f99821c11a6024035ae9873cd00d3f"} Sep 09 21:15:25.960334 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.959881 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" event={"ID":"dcdd769d-2f06-4103-bd71-9e53a85d0faf","Type":"ContainerStarted","Data":"34ab9c3db7bf4353b4a26f1ce3c7622d494e7dc9490f6f83d9ba294c366c9000"} Sep 09 21:15:25.987879 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.987854 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:25.988755 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:25.988715 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" podStartSLOduration=1.9886973289999998 podStartE2EDuration="1.988697329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:25.987815439 +0000 UTC m=+828.801665588" watchObservedRunningTime="2026-09-09 21:15:25.988697329 +0000 UTC m=+828.802547478" Sep 09 21:15:26.993256 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:26.993222 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:27.969959 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:27.969924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:15:27.971028 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:15:27.971009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:16:37.625288 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:37.625207 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:16:37.626381 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:37.626357 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:16:54.570790 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.570753 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz"] Sep 09 21:16:54.573147 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.573132 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:54.577011 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.576989 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zd5t2\"" Sep 09 21:16:54.577175 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.577152 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 09 21:16:54.596073 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.596049 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz"] Sep 09 21:16:54.662802 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.662770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d0cd34-af8a-4011-87f8-acc3c1b10d77-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9hbcz\" (UID: \"96d0cd34-af8a-4011-87f8-acc3c1b10d77\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:54.662964 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.662839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkhvv\" (UniqueName: \"kubernetes.io/projected/96d0cd34-af8a-4011-87f8-acc3c1b10d77-kube-api-access-jkhvv\") pod \"payload-pre-processing-59fbc47bf8-9hbcz\" (UID: \"96d0cd34-af8a-4011-87f8-acc3c1b10d77\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:54.764285 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.764245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkhvv\" (UniqueName: \"kubernetes.io/projected/96d0cd34-af8a-4011-87f8-acc3c1b10d77-kube-api-access-jkhvv\") pod \"payload-pre-processing-59fbc47bf8-9hbcz\" (UID: \"96d0cd34-af8a-4011-87f8-acc3c1b10d77\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:54.764488 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.764387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d0cd34-af8a-4011-87f8-acc3c1b10d77-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9hbcz\" (UID: \"96d0cd34-af8a-4011-87f8-acc3c1b10d77\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:54.765312 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.765283 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d0cd34-af8a-4011-87f8-acc3c1b10d77-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-9hbcz\" (UID: \"96d0cd34-af8a-4011-87f8-acc3c1b10d77\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:54.780966 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.780930 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkhvv\" (UniqueName: \"kubernetes.io/projected/96d0cd34-af8a-4011-87f8-acc3c1b10d77-kube-api-access-jkhvv\") pod \"payload-pre-processing-59fbc47bf8-9hbcz\" (UID: \"96d0cd34-af8a-4011-87f8-acc3c1b10d77\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:54.884011 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:54.883922 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:55.161661 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:55.161566 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz"] Sep 09 21:16:55.176708 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:16:55.176679 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d0cd34_af8a_4011_87f8_acc3c1b10d77.slice/crio-e7b74485c4daa20967303e8af642b43f84d85d79333bed7f3db59c08453fbed1 WatchSource:0}: Error finding container e7b74485c4daa20967303e8af642b43f84d85d79333bed7f3db59c08453fbed1: Status 404 returned error can't find the container with id e7b74485c4daa20967303e8af642b43f84d85d79333bed7f3db59c08453fbed1 Sep 09 21:16:55.178686 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:55.178669 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:16:55.362845 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:55.362790 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" event={"ID":"96d0cd34-af8a-4011-87f8-acc3c1b10d77","Type":"ContainerStarted","Data":"e7b74485c4daa20967303e8af642b43f84d85d79333bed7f3db59c08453fbed1"} Sep 09 21:16:58.389487 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:58.389396 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" event={"ID":"96d0cd34-af8a-4011-87f8-acc3c1b10d77","Type":"ContainerStarted","Data":"4c58b3b2898d03105756905168ec34eccaa61fde6651d0c37b18b0ecc0a91670"} Sep 09 21:16:58.389865 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:58.389551 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:16:58.410092 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:16:58.410048 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" podStartSLOduration=1.503425407 podStartE2EDuration="4.41003381s" totalImagesPullingTime="2.906608403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:58.408796125 +0000 UTC m=+921.222646277" watchObservedRunningTime="2026-09-09 21:16:58.41003381 +0000 UTC m=+921.223884007" Sep 09 21:17:03.415034 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:03.415002 2578 generic.go:356] "Generic (PLEG): container finished" podID="dcdd769d-2f06-4103-bd71-9e53a85d0faf" containerID="948a7c72e4d40aa65cfa2dd28e02a86b19f99821c11a6024035ae9873cd00d3f" exitCode=0 Sep 09 21:17:03.415458 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:03.415052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" event={"ID":"dcdd769d-2f06-4103-bd71-9e53a85d0faf","Type":"ContainerDied","Data":"948a7c72e4d40aa65cfa2dd28e02a86b19f99821c11a6024035ae9873cd00d3f"} Sep 09 21:17:03.415458 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:03.415416 2578 scope.go:117] "RemoveContainer" containerID="948a7c72e4d40aa65cfa2dd28e02a86b19f99821c11a6024035ae9873cd00d3f" Sep 09 21:17:03.416379 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:03.416346 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:17:03.416494 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:03.416445 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:17:03.416556 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:03.416492 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Sep 09 21:17:04.421711 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:04.421675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" event={"ID":"dcdd769d-2f06-4103-bd71-9e53a85d0faf","Type":"ContainerStarted","Data":"858e3e43a09071be0fd5d9658ee92ca6c9b7040e0c77a44f599846f62e1dda17"} Sep 09 21:17:04.988490 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:04.988452 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:17:04.988749 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:04.988731 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-trpxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Sep 09 21:17:04.988793 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:04.988773 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" podUID="dcdd769d-2f06-4103-bd71-9e53a85d0faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Sep 09 21:17:05.988396 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:05.988355 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-trpxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Sep 09 21:17:05.988872 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:05.988429 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" podUID="dcdd769d-2f06-4103-bd71-9e53a85d0faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Sep 09 21:17:06.988897 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:06.988857 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-trpxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Sep 09 21:17:06.989299 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:06.988924 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" podUID="dcdd769d-2f06-4103-bd71-9e53a85d0faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Sep 09 21:17:07.988383 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:07.988344 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-trpxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Sep 09 21:17:07.988588 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:07.988409 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" podUID="dcdd769d-2f06-4103-bd71-9e53a85d0faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Sep 09 21:17:08.988391 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:08.988355 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-trpxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Sep 09 21:17:08.988794 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:08.988409 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" podUID="dcdd769d-2f06-4103-bd71-9e53a85d0faf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Sep 09 21:17:09.397270 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:09.397155 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-9hbcz" Sep 09 21:17:09.992335 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:09.992307 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:17:09.992714 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:09.992559 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:17:09.993327 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:09.993304 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-trpxs" Sep 09 21:17:36.495755 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.495721 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h"] Sep 09 21:17:36.504397 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.504372 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.508498 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.508479 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4pf52\"" Sep 09 21:17:36.508743 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.508725 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 21:17:36.509429 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.509407 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 09 21:17:36.510658 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.510638 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 21:17:36.512412 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.512394 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h"] Sep 09 21:17:36.636604 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.636575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.636802 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.636610 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.636802 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.636710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.636802 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.636756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v6hn\" (UniqueName: \"kubernetes.io/projected/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-kube-api-access-7v6hn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.636966 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.636814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.636966 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.636872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.737946 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.737916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.737946 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.737948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.738218 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.737982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.738218 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.738003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v6hn\" (UniqueName: \"kubernetes.io/projected/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-kube-api-access-7v6hn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.738218 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.738125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.738218 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.738182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.738432 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.738309 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.738432 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.738376 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.738502 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.738460 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.740390 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.740371 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.740571 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.740552 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.747912 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.747855 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v6hn\" (UniqueName: \"kubernetes.io/projected/6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6-kube-api-access-7v6hn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h\" (UID: \"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:36.815779 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:36.815745 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:37.088544 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:37.088519 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h"] Sep 09 21:17:37.090477 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:17:37.090450 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4096d2_7ace_4401_bf0a_c2c5b39ee3a6.slice/crio-e285d2fcef55b73449db1841fc09ec03773388fee28e3dcda3e0bd9ee4bf6e86 WatchSource:0}: Error finding container e285d2fcef55b73449db1841fc09ec03773388fee28e3dcda3e0bd9ee4bf6e86: Status 404 returned error can't find the container with id e285d2fcef55b73449db1841fc09ec03773388fee28e3dcda3e0bd9ee4bf6e86 Sep 09 21:17:37.573460 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:37.573420 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" event={"ID":"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6","Type":"ContainerStarted","Data":"e285d2fcef55b73449db1841fc09ec03773388fee28e3dcda3e0bd9ee4bf6e86"} Sep 09 21:17:38.579620 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:38.579584 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" event={"ID":"6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6","Type":"ContainerStarted","Data":"5ee0c1a394af5dbb6680e96cf24161394244c67e547e29a90e83a89dfaa14e2f"} Sep 09 21:17:38.603882 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:38.603835 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" podStartSLOduration=1.672048585 podStartE2EDuration="2.603821106s" totalImagesPullingTime="931.772521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:17:38.601744787 +0000 UTC m=+961.415594951" watchObservedRunningTime="2026-09-09 21:17:38.603821106 +0000 UTC m=+961.417671254" Sep 09 21:17:44.314397 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.314364 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n"] Sep 09 21:17:44.316813 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.316797 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.319600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.319578 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 09 21:17:44.329031 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.329006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n"] Sep 09 21:17:44.403704 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.403666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.403704 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.403702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900ec71d-4bd7-4913-b952-e0c5aaf3e820-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.403904 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.403807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.403904 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.403867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.403983 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.403912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.403983 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.403943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8vd2\" (UniqueName: \"kubernetes.io/projected/900ec71d-4bd7-4913-b952-e0c5aaf3e820-kube-api-access-j8vd2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.504628 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.504595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.504798 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.504649 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.504798 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.504684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.504798 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.504708 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8vd2\" (UniqueName: \"kubernetes.io/projected/900ec71d-4bd7-4913-b952-e0c5aaf3e820-kube-api-access-j8vd2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.504798 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.504753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.504798 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.504778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900ec71d-4bd7-4913-b952-e0c5aaf3e820-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.505079 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.505052 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.505131 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.505090 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.505178 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.505159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.507064 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.507040 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/900ec71d-4bd7-4913-b952-e0c5aaf3e820-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.507148 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.507128 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/900ec71d-4bd7-4913-b952-e0c5aaf3e820-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.518372 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.518356 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8vd2\" (UniqueName: \"kubernetes.io/projected/900ec71d-4bd7-4913-b952-e0c5aaf3e820-kube-api-access-j8vd2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9m9n\" (UID: \"900ec71d-4bd7-4913-b952-e0c5aaf3e820\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.626883 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.626798 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:44.904472 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:44.904398 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n"] Sep 09 21:17:44.908596 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:17:44.908571 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900ec71d_4bd7_4913_b952_e0c5aaf3e820.slice/crio-3e044373cd7e312755e1f0206e24e38fafe02fb667790a174d18acbc0fdee849 WatchSource:0}: Error finding container 3e044373cd7e312755e1f0206e24e38fafe02fb667790a174d18acbc0fdee849: Status 404 returned error can't find the container with id 3e044373cd7e312755e1f0206e24e38fafe02fb667790a174d18acbc0fdee849 Sep 09 21:17:45.615529 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:45.615499 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" event={"ID":"900ec71d-4bd7-4913-b952-e0c5aaf3e820","Type":"ContainerStarted","Data":"cc663b1596be1d3b72c767fd41e6ad2b076811ba22517562cd365ab0731e625f"} Sep 09 21:17:45.615882 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:45.615547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" event={"ID":"900ec71d-4bd7-4913-b952-e0c5aaf3e820","Type":"ContainerStarted","Data":"3e044373cd7e312755e1f0206e24e38fafe02fb667790a174d18acbc0fdee849"} Sep 09 21:17:45.638283 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:45.638241 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" podStartSLOduration=1.488692145 podStartE2EDuration="1.638225448s" totalImagesPullingTime="149.533303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:17:45.636865294 +0000 UTC m=+968.450715442" watchObservedRunningTime="2026-09-09 21:17:45.638225448 +0000 UTC m=+968.452075596" Sep 09 21:17:46.816703 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:46.816637 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:46.816703 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:46.816693 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:46.829102 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:46.829074 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:47.637628 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:47.637599 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h" Sep 09 21:17:54.626972 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:54.626939 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:54.627455 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:54.626997 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:54.639280 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:54.639256 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:17:54.676279 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:17:54.676257 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9m9n" Sep 09 21:18:13.204923 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.204886 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf"] Sep 09 21:18:13.214282 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.214258 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.216794 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.216775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 09 21:18:13.226134 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.226110 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf"] Sep 09 21:18:13.364511 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.364478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.364694 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.364593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.364762 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.364699 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.364762 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.364731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9j55\" (UniqueName: \"kubernetes.io/projected/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-kube-api-access-v9j55\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.364861 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.364771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.364861 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.364795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.465754 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.465727 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.465884 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.465768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.465884 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.465806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.465884 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.465822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9j55\" (UniqueName: \"kubernetes.io/projected/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-kube-api-access-v9j55\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.465884 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.465842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.465884 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.465856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.466243 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.466191 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.466243 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.466233 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.466325 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.466284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.468348 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.468323 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.468542 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.468524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.475246 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.475225 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9j55\" (UniqueName: \"kubernetes.io/projected/b1bd9d22-974c-4bed-b68c-9fb3921b6ce8-kube-api-access-v9j55\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf\" (UID: \"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.526031 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.526004 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:13.814164 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:13.810487 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf"] Sep 09 21:18:13.815823 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:18:13.815795 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1bd9d22_974c_4bed_b68c_9fb3921b6ce8.slice/crio-68e110195d92f9122d3b641bf396eef3bacc4f1ddf34cbf72c643d93dffcc509 WatchSource:0}: Error finding container 68e110195d92f9122d3b641bf396eef3bacc4f1ddf34cbf72c643d93dffcc509: Status 404 returned error can't find the container with id 68e110195d92f9122d3b641bf396eef3bacc4f1ddf34cbf72c643d93dffcc509 Sep 09 21:18:14.750281 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:14.750249 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" containerID="fa3f247925264f2189ac2b9930307e140e4fa20ae2806d5ce26115b582377bd0" exitCode=0 Sep 09 21:18:14.750757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:14.750342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" event={"ID":"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8","Type":"ContainerDied","Data":"fa3f247925264f2189ac2b9930307e140e4fa20ae2806d5ce26115b582377bd0"} Sep 09 21:18:14.750757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:14.750391 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" event={"ID":"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8","Type":"ContainerStarted","Data":"68e110195d92f9122d3b641bf396eef3bacc4f1ddf34cbf72c643d93dffcc509"} Sep 09 21:18:14.750757 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:14.750611 2578 scope.go:117] "RemoveContainer" containerID="fa3f247925264f2189ac2b9930307e140e4fa20ae2806d5ce26115b582377bd0" Sep 09 21:18:15.757123 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:15.757089 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" containerID="171c8a79b2a660c0c72b949a5b57f2406993a5dfad44059f9f1d1bef19b90468" exitCode=0 Sep 09 21:18:15.757571 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:15.757176 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" event={"ID":"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8","Type":"ContainerDied","Data":"171c8a79b2a660c0c72b949a5b57f2406993a5dfad44059f9f1d1bef19b90468"} Sep 09 21:18:15.757571 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:15.757262 2578 scope.go:117] "RemoveContainer" containerID="fa3f247925264f2189ac2b9930307e140e4fa20ae2806d5ce26115b582377bd0" Sep 09 21:18:15.757571 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:15.757563 2578 scope.go:117] "RemoveContainer" containerID="171c8a79b2a660c0c72b949a5b57f2406993a5dfad44059f9f1d1bef19b90468" Sep 09 21:18:15.757822 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:18:15.757796 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:18:16.763461 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:16.763438 2578 scope.go:117] "RemoveContainer" containerID="171c8a79b2a660c0c72b949a5b57f2406993a5dfad44059f9f1d1bef19b90468" Sep 09 21:18:16.763816 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:18:16.763637 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:18:23.526741 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:23.526706 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:23.527130 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:23.526765 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:23.527130 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:23.526786 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:23.527130 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:23.526798 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:23.527262 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:23.527180 2578 scope.go:117] "RemoveContainer" containerID="171c8a79b2a660c0c72b949a5b57f2406993a5dfad44059f9f1d1bef19b90468" Sep 09 21:18:23.527393 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:18:23.527375 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:18:35.682400 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:35.682371 2578 scope.go:117] "RemoveContainer" containerID="171c8a79b2a660c0c72b949a5b57f2406993a5dfad44059f9f1d1bef19b90468" Sep 09 21:18:36.865703 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:36.865671 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" containerID="6d2c373046c0ce055eb04c1b8b1a54abc211d6b8048e91e05426b6342b01f072" exitCode=0 Sep 09 21:18:36.866161 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:36.865738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" event={"ID":"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8","Type":"ContainerDied","Data":"6d2c373046c0ce055eb04c1b8b1a54abc211d6b8048e91e05426b6342b01f072"} Sep 09 21:18:36.866161 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:36.865796 2578 scope.go:117] "RemoveContainer" containerID="171c8a79b2a660c0c72b949a5b57f2406993a5dfad44059f9f1d1bef19b90468" Sep 09 21:18:36.866306 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:36.866220 2578 scope.go:117] "RemoveContainer" containerID="6d2c373046c0ce055eb04c1b8b1a54abc211d6b8048e91e05426b6342b01f072" Sep 09 21:18:36.866459 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:18:36.866439 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:18:43.526762 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:43.526733 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:43.527166 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:43.526784 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:43.527166 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:43.526799 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:43.527166 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:43.526814 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:18:43.527304 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:43.527223 2578 scope.go:117] "RemoveContainer" containerID="6d2c373046c0ce055eb04c1b8b1a54abc211d6b8048e91e05426b6342b01f072" Sep 09 21:18:43.527428 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:18:43.527408 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:18:57.685745 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:57.685709 2578 scope.go:117] "RemoveContainer" containerID="6d2c373046c0ce055eb04c1b8b1a54abc211d6b8048e91e05426b6342b01f072" Sep 09 21:18:57.963991 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:57.963966 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" containerID="2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99" exitCode=0 Sep 09 21:18:57.964115 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:57.964044 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" event={"ID":"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8","Type":"ContainerDied","Data":"2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99"} Sep 09 21:18:57.964186 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:57.964126 2578 scope.go:117] "RemoveContainer" containerID="6d2c373046c0ce055eb04c1b8b1a54abc211d6b8048e91e05426b6342b01f072" Sep 09 21:18:57.964521 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:18:57.964500 2578 scope.go:117] "RemoveContainer" containerID="2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99" Sep 09 21:18:57.964745 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:18:57.964725 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:19:03.526456 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:03.526415 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:03.526861 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:03.526508 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:03.526861 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:03.526524 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:03.526861 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:03.526538 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:03.526959 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:03.526916 2578 scope.go:117] "RemoveContainer" containerID="2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99" Sep 09 21:19:03.527138 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:19:03.527118 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:19:14.682600 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:14.682570 2578 scope.go:117] "RemoveContainer" containerID="2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99" Sep 09 21:19:14.682992 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:19:14.682821 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:19:28.682180 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:28.682100 2578 scope.go:117] "RemoveContainer" containerID="2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99" Sep 09 21:19:28.682656 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:19:28.682343 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:19:40.682248 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:40.682217 2578 scope.go:117] "RemoveContainer" containerID="2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99" Sep 09 21:19:41.177134 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:41.177102 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" exitCode=0 Sep 09 21:19:41.177134 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:41.177136 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" event={"ID":"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8","Type":"ContainerDied","Data":"6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c"} Sep 09 21:19:41.177370 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:41.177184 2578 scope.go:117] "RemoveContainer" containerID="2ffaf9c52ef919abd9886d0f22a51cf0ebba7a65b5417467d46645540ca0ac99" Sep 09 21:19:41.177550 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:41.177530 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:19:41.177778 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:19:41.177754 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:19:43.526262 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:43.526221 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:43.526652 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:43.526287 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:43.526652 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:43.526304 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:43.526652 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:43.526317 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:19:43.526652 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:43.526603 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:19:43.526779 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:19:43.526753 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:19:54.682231 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:19:54.682187 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:19:54.682633 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:19:54.682371 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:20:06.682480 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:20:06.682406 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:20:06.682852 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:20:06.682572 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:20:21.688403 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:20:21.688374 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:20:21.688803 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:20:21.688546 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:20:32.681985 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:20:32.681956 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:20:32.682415 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:20:32.682133 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:20:43.682512 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:20:43.682476 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:20:43.683107 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:20:43.682683 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:20:58.682517 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:20:58.682440 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:20:58.682957 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:20:58.682655 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:21:09.682801 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:09.682771 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:21:10.599982 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:10.599953 2578 generic.go:356] "Generic (PLEG): container finished" podID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" exitCode=0 Sep 09 21:21:10.600181 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:10.600024 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" event={"ID":"b1bd9d22-974c-4bed-b68c-9fb3921b6ce8","Type":"ContainerDied","Data":"f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493"} Sep 09 21:21:10.600181 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:10.600078 2578 scope.go:117] "RemoveContainer" containerID="6d9c82b9ebb159d3b6774288b352f475c090031f08138ad3e5750ae0ddde826c" Sep 09 21:21:10.600515 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:10.600492 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:21:10.600735 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:21:10.600717 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:21:13.526363 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:13.526326 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:21:13.526739 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:13.526409 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:21:13.526739 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:13.526448 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:21:13.526739 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:13.526461 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" Sep 09 21:21:13.526739 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:13.526714 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:21:13.526903 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:21:13.526884 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:21:13.616993 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:13.616969 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:21:13.617160 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:21:13.617140 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:21:25.682215 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:25.682167 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:21:25.682631 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:21:25.682401 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:21:36.682574 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:36.682546 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:21:36.683053 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:21:36.682723 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:21:37.656830 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:37.656805 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:21:37.660818 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:37.660795 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:21:51.688154 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:21:51.688115 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:21:51.688564 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:21:51.688335 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:22:06.682642 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:06.682615 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:22:06.684982 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:22:06.682803 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:22:08.791829 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:08.791798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_b1bd9d22-974c-4bed-b68c-9fb3921b6ce8/main/5.log" Sep 09 21:22:08.919975 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:08.919943 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_b1bd9d22-974c-4bed-b68c-9fb3921b6ce8/main/5.log" Sep 09 21:22:16.303654 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:16.303622 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hprnw_c0bc754e-9e45-42f6-8404-536aa2e3ed6f/server/0.log" Sep 09 21:22:16.422239 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:16.422212 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vljx6_3ce49693-854d-48ca-9682-04be30f5d408/manager/0.log" Sep 09 21:22:16.551306 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:16.551278 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-78d27_8a8a4848-38dd-42fb-ae89-9b49e201d5a6/manager/0.log" Sep 09 21:22:16.559637 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:16.559570 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-78d27_8a8a4848-38dd-42fb-ae89-9b49e201d5a6/copy-manifests/0.log" Sep 09 21:22:17.684789 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:17.684758 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:22:17.685178 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:22:17.684936 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:22:19.326406 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:19.326375 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg_67b1da2e-49a6-4dd7-8501-f3f45e7fabf1/istio-proxy/0.log" Sep 09 21:22:19.450073 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:19.450044 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r7tht_59630cf3-f4de-49ba-8f47-37c6922a5e6f/discovery/0.log" Sep 09 21:22:19.824086 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:19.824059 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-trpxs_dcdd769d-2f06-4103-bd71-9e53a85d0faf/istio-proxy/1.log" Sep 09 21:22:19.952318 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:19.952289 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9hbcz_96d0cd34-af8a-4011-87f8-acc3c1b10d77/payload-pre-processing/0.log" Sep 09 21:22:20.785147 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:20.785113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-47c8h_6f4096d2-7ace-4401-bf0a-c2c5b39ee3a6/main/0.log" Sep 09 21:22:20.920807 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:20.920776 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-p9m9n_900ec71d-4bd7-4913-b952-e0c5aaf3e820/main/0.log" Sep 09 21:22:21.178140 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:21.178062 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_b1bd9d22-974c-4bed-b68c-9fb3921b6ce8/main/5.log" Sep 09 21:22:29.682437 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:29.682363 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:22:29.682833 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:22:29.682555 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:22:31.300839 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:31.300800 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vcxpp_6c202f57-bbf0-4eef-8fbc-8c8f2dd107a5/global-pull-secret-syncer/0.log" Sep 09 21:22:31.442085 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:31.442058 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gvcdp_109f2c1a-0905-49d5-bb38-092b9c9130a5/konnectivity-agent/0.log" Sep 09 21:22:31.526048 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:31.526017 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-18.ec2.internal_0559b432e0a8e8090650f241aa697b78/haproxy/0.log" Sep 09 21:22:37.170134 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.170096 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_735f6f0f-a247-4302-bb56-7c4de81a2176/alertmanager/0.log" Sep 09 21:22:37.199406 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.199377 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_735f6f0f-a247-4302-bb56-7c4de81a2176/config-reloader/0.log" Sep 09 21:22:37.232158 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.232135 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_735f6f0f-a247-4302-bb56-7c4de81a2176/kube-rbac-proxy-web/0.log" Sep 09 21:22:37.276401 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.276384 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_735f6f0f-a247-4302-bb56-7c4de81a2176/kube-rbac-proxy/0.log" Sep 09 21:22:37.311190 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.311172 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_735f6f0f-a247-4302-bb56-7c4de81a2176/kube-rbac-proxy-metric/0.log" Sep 09 21:22:37.342592 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.342573 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_735f6f0f-a247-4302-bb56-7c4de81a2176/prom-label-proxy/0.log" Sep 09 21:22:37.368563 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.368539 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_735f6f0f-a247-4302-bb56-7c4de81a2176/init-config-reloader/0.log" Sep 09 21:22:37.457140 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.457069 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-z8p25_9178b7a9-1800-443a-aa84-9b8d54687c22/kube-state-metrics/0.log" Sep 09 21:22:37.481930 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.481904 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-z8p25_9178b7a9-1800-443a-aa84-9b8d54687c22/kube-rbac-proxy-main/0.log" Sep 09 21:22:37.512291 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.512268 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-z8p25_9178b7a9-1800-443a-aa84-9b8d54687c22/kube-rbac-proxy-self/0.log" Sep 09 21:22:37.588695 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.588665 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-pbsmg_6cf17b4a-d499-43e7-9595-87b891bbab5a/monitoring-plugin/0.log" Sep 09 21:22:37.734664 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.734640 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pg6zh_f456c1db-4b9b-4334-82e1-9317eeb17180/node-exporter/0.log" Sep 09 21:22:37.766991 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.766971 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pg6zh_f456c1db-4b9b-4334-82e1-9317eeb17180/kube-rbac-proxy/0.log" Sep 09 21:22:37.792719 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:37.792701 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pg6zh_f456c1db-4b9b-4334-82e1-9317eeb17180/init-textfile/0.log" Sep 09 21:22:38.140365 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.140292 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa50c80b-9513-498f-b7d9-3a0216a246d4/prometheus/0.log" Sep 09 21:22:38.188599 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.188575 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa50c80b-9513-498f-b7d9-3a0216a246d4/config-reloader/0.log" Sep 09 21:22:38.221614 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.221598 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa50c80b-9513-498f-b7d9-3a0216a246d4/thanos-sidecar/0.log" Sep 09 21:22:38.250747 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.250719 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa50c80b-9513-498f-b7d9-3a0216a246d4/kube-rbac-proxy-web/0.log" Sep 09 21:22:38.286117 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.286093 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa50c80b-9513-498f-b7d9-3a0216a246d4/kube-rbac-proxy/0.log" Sep 09 21:22:38.322367 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.322350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa50c80b-9513-498f-b7d9-3a0216a246d4/kube-rbac-proxy-thanos/0.log" Sep 09 21:22:38.352017 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.351998 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_aa50c80b-9513-498f-b7d9-3a0216a246d4/init-config-reloader/0.log" Sep 09 21:22:38.472110 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.472089 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-6fp9f_e89d8e1d-d6fc-4236-8b4b-de98a07b1555/prometheus-operator-admission-webhook/0.log" Sep 09 21:22:38.527342 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.527318 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-785db4f4fc-kzj5f_12327d20-457a-493c-9073-639b020da874/telemeter-client/0.log" Sep 09 21:22:38.561162 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.561141 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-785db4f4fc-kzj5f_12327d20-457a-493c-9073-639b020da874/reload/0.log" Sep 09 21:22:38.604681 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:38.604652 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-785db4f4fc-kzj5f_12327d20-457a-493c-9073-639b020da874/kube-rbac-proxy/0.log" Sep 09 21:22:41.149878 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.149850 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/1.log" Sep 09 21:22:41.157126 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.157102 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-2bt64_3063cd92-a3aa-4008-a6b1-90010561a7d3/console-operator/2.log" Sep 09 21:22:41.376139 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.376101 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4"] Sep 09 21:22:41.379622 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.379603 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.382489 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.382468 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dvsfq\"/\"openshift-service-ca.crt\"" Sep 09 21:22:41.382756 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.382736 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dvsfq\"/\"kube-root-ca.crt\"" Sep 09 21:22:41.383473 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.383457 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dvsfq\"/\"default-dockercfg-rmncw\"" Sep 09 21:22:41.390331 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.390308 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4"] Sep 09 21:22:41.496776 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.496737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dljpg\" (UniqueName: \"kubernetes.io/projected/905c7914-ef52-4d85-b120-7e7342e8c13f-kube-api-access-dljpg\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.496968 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.496823 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-podres\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.496968 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.496862 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-lib-modules\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.496968 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.496946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-sys\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.497128 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.496988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-proc\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.597820 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.597787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-podres\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.597986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.597827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-lib-modules\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.597986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.597879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-sys\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.597986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.597896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-proc\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.597986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.597961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dljpg\" (UniqueName: \"kubernetes.io/projected/905c7914-ef52-4d85-b120-7e7342e8c13f-kube-api-access-dljpg\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.597986 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.597965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-podres\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.598268 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.597990 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-sys\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.598268 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.598027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-proc\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.598268 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.598046 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/905c7914-ef52-4d85-b120-7e7342e8c13f-lib-modules\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.608283 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.608256 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dljpg\" (UniqueName: \"kubernetes.io/projected/905c7914-ef52-4d85-b120-7e7342e8c13f-kube-api-access-dljpg\") pod \"perf-node-gather-daemonset-mlfq4\" (UID: \"905c7914-ef52-4d85-b120-7e7342e8c13f\") " pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.691496 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.691477 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:41.967985 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:41.967955 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4"] Sep 09 21:22:41.971295 ip-10-0-133-18 kubenswrapper[2578]: W0909 21:22:41.971268 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod905c7914_ef52_4d85_b120_7e7342e8c13f.slice/crio-d05ef14adc98ccf5b544d1732242d080370d2416e8abdc89cc3f9bb18025a214 WatchSource:0}: Error finding container d05ef14adc98ccf5b544d1732242d080370d2416e8abdc89cc3f9bb18025a214: Status 404 returned error can't find the container with id d05ef14adc98ccf5b544d1732242d080370d2416e8abdc89cc3f9bb18025a214 Sep 09 21:22:42.029122 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:42.029093 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" event={"ID":"905c7914-ef52-4d85-b120-7e7342e8c13f","Type":"ContainerStarted","Data":"d05ef14adc98ccf5b544d1732242d080370d2416e8abdc89cc3f9bb18025a214"} Sep 09 21:22:42.507060 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:42.507033 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-vpp7p_0e64348c-0902-4b86-ad8e-1a2300b3a555/volume-data-source-validator/0.log" Sep 09 21:22:42.682330 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:42.682300 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:22:42.682553 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:22:42.682530 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:22:43.036688 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:43.036658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" event={"ID":"905c7914-ef52-4d85-b120-7e7342e8c13f","Type":"ContainerStarted","Data":"0102214b62ef553f5e45d5715a59064be36ff1ebf752dcccd3bffd3f97abf5ef"} Sep 09 21:22:43.036849 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:43.036710 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:43.057468 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:43.057425 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" podStartSLOduration=2.057410656 podStartE2EDuration="2.057410656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:22:43.055509845 +0000 UTC m=+1265.869359997" watchObservedRunningTime="2026-09-09 21:22:43.057410656 +0000 UTC m=+1265.871260843" Sep 09 21:22:43.811807 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:43.811780 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zs7k6_90fcbf0d-a066-48c7-a685-4d51e021b67f/dns/0.log" Sep 09 21:22:43.842059 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:43.842035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zs7k6_90fcbf0d-a066-48c7-a685-4d51e021b67f/kube-rbac-proxy/0.log" Sep 09 21:22:43.937266 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:43.937242 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rjhvd_7f2d556d-c2dc-4e5f-b243-b320d11c24d4/dns-node-resolver/0.log" Sep 09 21:22:44.535402 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:44.535371 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-787f6d466-dsx87_4f960ddd-7f52-43b4-a121-7a14d7d13e9f/registry/0.log" Sep 09 21:22:44.564091 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:44.564059 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7lm7d_85e4a129-3ede-4946-b4a2-2d86b351df38/node-ca/0.log" Sep 09 21:22:45.801157 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:45.801116 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg_67b1da2e-49a6-4dd7-8501-f3f45e7fabf1/istio-proxy/0.log" Sep 09 21:22:45.857028 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:45.856985 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r7tht_59630cf3-f4de-49ba-8f47-37c6922a5e6f/discovery/0.log" Sep 09 21:22:45.948299 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:45.948277 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-trpxs_dcdd769d-2f06-4103-bd71-9e53a85d0faf/istio-proxy/1.log" Sep 09 21:22:45.948664 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:45.948649 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-trpxs_dcdd769d-2f06-4103-bd71-9e53a85d0faf/istio-proxy/0.log" Sep 09 21:22:45.975004 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:45.974981 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9hbcz_96d0cd34-af8a-4011-87f8-acc3c1b10d77/payload-pre-processing/0.log" Sep 09 21:22:46.727466 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:46.727435 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lld6z_6951edbb-68bf-4b11-8bbc-33b960a5c4d5/serve-healthcheck-canary/0.log" Sep 09 21:22:47.568219 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:47.568172 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bxz7h_0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa/kube-rbac-proxy/0.log" Sep 09 21:22:47.599038 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:47.599007 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bxz7h_0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa/exporter/0.log" Sep 09 21:22:47.632544 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:47.632497 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bxz7h_0973e706-c8f1-4cdb-b5c2-69ce3a5b77fa/extractor/0.log" Sep 09 21:22:49.051561 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:49.051531 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dvsfq/perf-node-gather-daemonset-mlfq4" Sep 09 21:22:50.226456 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:50.226429 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hprnw_c0bc754e-9e45-42f6-8404-536aa2e3ed6f/server/0.log" Sep 09 21:22:50.291177 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:50.291147 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vljx6_3ce49693-854d-48ca-9682-04be30f5d408/manager/0.log" Sep 09 21:22:50.328673 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:50.328647 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-78d27_8a8a4848-38dd-42fb-ae89-9b49e201d5a6/manager/0.log" Sep 09 21:22:50.374247 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:50.374227 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59685ff6b-78d27_8a8a4848-38dd-42fb-ae89-9b49e201d5a6/copy-manifests/0.log" Sep 09 21:22:51.522946 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:51.522921 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg_67b1da2e-49a6-4dd7-8501-f3f45e7fabf1/istio-proxy/0.log" Sep 09 21:22:51.562617 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:51.562589 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r7tht_59630cf3-f4de-49ba-8f47-37c6922a5e6f/discovery/0.log" Sep 09 21:22:51.642414 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:51.642390 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-trpxs_dcdd769d-2f06-4103-bd71-9e53a85d0faf/istio-proxy/1.log" Sep 09 21:22:51.642764 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:51.642746 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-trpxs_dcdd769d-2f06-4103-bd71-9e53a85d0faf/istio-proxy/0.log" Sep 09 21:22:51.657605 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:51.657590 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9hbcz_96d0cd34-af8a-4011-87f8-acc3c1b10d77/payload-pre-processing/0.log" Sep 09 21:22:52.009965 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:52.009940 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57b847cbcb-wkqs8_e39ac3b4-fcb4-4096-bc86-9f563c68d1dc/manager/0.log" Sep 09 21:22:53.044463 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:53.044432 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-5q2rg_67b1da2e-49a6-4dd7-8501-f3f45e7fabf1/istio-proxy/0.log" Sep 09 21:22:53.100461 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:53.100390 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r7tht_59630cf3-f4de-49ba-8f47-37c6922a5e6f/discovery/0.log" Sep 09 21:22:53.189826 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:53.189799 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-trpxs_dcdd769d-2f06-4103-bd71-9e53a85d0faf/istio-proxy/1.log" Sep 09 21:22:53.189981 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:53.189899 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-trpxs_dcdd769d-2f06-4103-bd71-9e53a85d0faf/istio-proxy/0.log" Sep 09 21:22:53.211801 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:53.211781 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-9hbcz_96d0cd34-af8a-4011-87f8-acc3c1b10d77/payload-pre-processing/0.log" Sep 09 21:22:54.681899 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:54.681825 2578 scope.go:117] "RemoveContainer" containerID="f1da5b27d2e18fe708ec389417771c8e65d0d199ca8d48bc9dd1aa29e4b3e493" Sep 09 21:22:54.682276 ip-10-0-133-18 kubenswrapper[2578]: E0909 21:22:54.682021 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf_llm(b1bd9d22-974c-4bed-b68c-9fb3921b6ce8)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-5jxcf" podUID="b1bd9d22-974c-4bed-b68c-9fb3921b6ce8" Sep 09 21:22:58.095144 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:58.095114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-xptdf_366c43e6-0530-4f81-a877-a726708279cd/migrator/0.log" Sep 09 21:22:58.133131 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:22:58.133101 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-xptdf_366c43e6-0530-4f81-a877-a726708279cd/graceful-termination/0.log" Sep 09 21:23:00.075827 ip-10-0-133-18 kubenswrapper[2578]: I0909 21:23:00.075800 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-989mt_00dbb388-31d8-4461-ba75-396770d7720e/kube-multus-additional-cni-plugins/0.log"