Jul 14 17:04:57.317897 ip-10-0-128-240 systemd[1]: Starting Kubernetes Kubelet... Jul 14 17:04:58.961028 ip-10-0-128-240 systemd[1]: Started Kubernetes Kubelet. Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.884495 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890561 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890579 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890583 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890587 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890590 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890593 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890596 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:04:58.985420 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890602 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890606 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890609 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890626 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890631 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890634 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890638 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890641 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890644 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890647 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890650 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890653 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890656 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890660 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890662 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890665 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890668 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890671 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890674 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:04:58.986883 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890676 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890682 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890687 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890691 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890695 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890699 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890702 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890705 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890708 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890711 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890715 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890718 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890721 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890724 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890727 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890729 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890732 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890736 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890738 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890741 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:04:58.989690 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890743 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890746 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890749 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890751 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890754 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890757 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890760 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890763 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890766 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890768 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890771 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890773 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890776 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890779 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890783 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890786 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890789 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890792 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890794 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890797 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:04:58.990773 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890800 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890803 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890806 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890809 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890812 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890815 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890817 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890820 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890823 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890827 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890830 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890832 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890835 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890838 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890840 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890845 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890848 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890851 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890854 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890857 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:04:58.994754 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890861 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890864 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890867 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890870 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890873 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890878 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890881 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890884 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890888 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890890 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890893 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890896 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.890899 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893282 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893292 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893297 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893301 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893305 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893308 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893311 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:04:58.995876 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893315 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893318 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893321 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893324 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893327 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893330 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893333 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893336 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893339 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893341 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893344 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893347 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893349 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893354 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893358 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893361 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893364 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893368 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893371 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:04:59.094003 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893375 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893377 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893380 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893383 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893386 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893388 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893391 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893395 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893397 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893400 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893403 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893406 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893409 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893411 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893414 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893416 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893419 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893422 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893426 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893430 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:04:59.095077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893433 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893436 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893440 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893442 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893446 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893448 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893451 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893454 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893458 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893463 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893466 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893469 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893472 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893476 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893479 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893482 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893485 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893488 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893492 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:04:59.104184 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893494 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893497 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893500 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893502 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893506 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893509 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893511 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893514 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893516 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893520 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893523 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893528 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893531 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893534 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893536 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893539 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893542 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893545 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893548 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893550 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:04:59.105366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893553 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893556 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893558 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893561 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893564 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893566 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893569 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893572 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893574 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893577 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893580 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893583 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893585 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.893588 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894044 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894050 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894054 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894057 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894060 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894063 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:04:59.106173 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894066 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894069 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894071 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894077 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894079 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894082 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894085 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894088 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894091 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894096 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894100 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894104 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894108 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894113 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894117 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894120 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894123 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894126 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:04:59.107742 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894130 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894133 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894135 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894139 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894143 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894146 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894149 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894152 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894154 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894157 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894160 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894164 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894166 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894169 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894173 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894176 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894180 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894185 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894188 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894191 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:04:59.108528 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894194 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894197 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894200 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894202 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894205 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894208 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894211 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894214 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894216 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894219 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894222 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894224 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894227 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894230 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894233 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894235 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894238 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894241 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894244 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894248 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:04:59.109720 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894250 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894253 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894256 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894259 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894268 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894272 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894274 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894277 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894281 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894292 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894294 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894297 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894300 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894303 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894305 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894308 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894311 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894314 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894316 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894319 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:04:59.110493 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894322 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894325 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894327 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894330 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894333 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894336 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894338 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894341 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894344 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894346 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894349 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894352 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894355 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894357 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.894360 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894446 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894454 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894482 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894487 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894491 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894496 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 17:04:59.111661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894502 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894507 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894510 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894514 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894517 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894521 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894524 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894528 2558 flags.go:64] FLAG: --cgroup-root="" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894531 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894534 2558 flags.go:64] FLAG: --client-ca-file="" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894537 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894541 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894549 2558 flags.go:64] FLAG: --cluster-domain="" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894552 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894556 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894559 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894563 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894567 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894570 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894574 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894577 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894580 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894584 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894587 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 17:04:59.114057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894590 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894593 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894598 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894601 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894604 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894608 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894627 2558 flags.go:64] FLAG: --enable-server="true" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894634 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894644 2558 flags.go:64] FLAG: --event-burst="100" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894649 2558 flags.go:64] FLAG: --event-qps="50" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894652 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894655 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894659 2558 flags.go:64] FLAG: --eviction-hard="" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894663 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894666 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894670 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894673 2558 flags.go:64] FLAG: --eviction-soft="" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894676 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894680 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894683 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894686 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894690 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894693 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894696 2558 flags.go:64] FLAG: --feature-gates="" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894700 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 17:04:59.114940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894703 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894707 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894710 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894714 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894717 2558 flags.go:64] FLAG: --help="false" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894721 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894724 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894727 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894730 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894734 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894737 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894741 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894744 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894747 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894752 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894757 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894761 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894764 2558 flags.go:64] FLAG: --kube-reserved="" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894767 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894770 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894773 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894776 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894779 2558 flags.go:64] FLAG: --lock-file="" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894782 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 17:04:59.115807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894786 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894789 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894796 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894799 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894802 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894805 2558 flags.go:64] FLAG: --logging-format="text" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894808 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894812 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894815 2558 flags.go:64] FLAG: --manifest-url="" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894830 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894835 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894838 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894843 2558 flags.go:64] FLAG: --max-pods="110" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894847 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894850 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894853 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894856 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894860 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894863 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894866 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894874 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894878 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894887 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 17:04:59.116628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894892 2558 flags.go:64] FLAG: --pod-cidr="" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894895 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894902 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894905 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894909 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894912 2558 flags.go:64] FLAG: --port="10250" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894915 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894918 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e857abee81ed3751" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894922 2558 flags.go:64] FLAG: --qos-reserved="" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894925 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894928 2558 flags.go:64] FLAG: --register-node="true" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894932 2558 flags.go:64] FLAG: --register-with-taints="" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894936 2558 flags.go:64] FLAG: --registry-burst="10" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894939 2558 flags.go:64] FLAG: --registry-qps="5" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894943 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894946 2558 flags.go:64] FLAG: --reserved-memory="" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894950 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894953 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894956 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894960 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894964 2558 flags.go:64] FLAG: --runonce="false" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894967 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894970 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894973 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894977 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 17:04:59.117397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894980 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894983 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894986 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894990 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894993 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.894996 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895001 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895010 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895016 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895020 2558 flags.go:64] FLAG: --system-cgroups="" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895023 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895029 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895032 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895035 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895043 2558 flags.go:64] FLAG: --tls-min-version="" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895046 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895049 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895052 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895055 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895059 2558 flags.go:64] FLAG: --v="2" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895063 2558 flags.go:64] FLAG: --version="false" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895067 2558 flags.go:64] FLAG: --vmodule="" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895072 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895075 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 17:04:59.118284 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895223 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895227 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895231 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895234 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895237 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895240 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895243 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895246 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895249 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895251 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895254 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895257 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895260 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895263 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895268 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895272 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895279 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895282 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895285 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:04:59.119093 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895288 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895291 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895294 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895297 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895300 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895304 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895307 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895310 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895313 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895316 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895318 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895321 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895324 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895327 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895330 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895333 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895335 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895338 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895341 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895344 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:04:59.119758 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895346 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895349 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895352 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895355 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895357 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895360 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895363 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895368 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895370 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895376 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895380 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895383 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895386 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895389 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895392 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895395 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895398 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895401 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895404 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895406 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:04:59.120424 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895409 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895412 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895415 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895417 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895420 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895423 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895426 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895429 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895432 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895434 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895437 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895440 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895443 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895446 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895448 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895451 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895454 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895457 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895459 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895464 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:04:59.121102 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895466 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895472 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895475 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895479 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895481 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895484 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895487 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895489 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895492 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895495 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895498 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895502 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895506 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895509 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895512 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895515 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895519 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895522 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895525 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:04:59.121767 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.895528 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.895534 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.903204 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.903226 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.904072 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.904086 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904155 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904161 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904185 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904190 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904194 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904198 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904201 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904204 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904207 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:04:59.122390 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904212 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904217 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904221 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904224 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904227 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904230 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904233 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904236 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904238 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904242 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904245 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904248 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904251 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904253 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904256 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904259 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904262 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904265 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904268 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904271 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:04:59.122910 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904275 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904278 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904282 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904285 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904288 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904291 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904294 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904297 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904299 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904302 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904305 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904308 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904311 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904313 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904317 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904319 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904322 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904325 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904329 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:04:59.123559 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904333 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904336 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904340 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904343 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904346 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904353 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904356 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904358 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904361 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904364 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904367 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904370 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904373 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904376 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904379 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904382 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904385 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904388 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904390 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904393 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:04:59.124204 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904396 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904399 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904402 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904406 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904410 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904413 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904416 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904420 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904424 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904427 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904430 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904433 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904436 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904439 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904442 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904444 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904447 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904452 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904454 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904457 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:04:59.124894 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904460 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904463 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904466 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904468 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904471 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904476 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904479 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904482 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904484 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904487 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904490 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.904496 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904642 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904648 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:04:59.125546 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904651 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904654 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904658 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904661 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904664 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904667 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904670 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904673 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904676 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904679 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904682 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904684 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904687 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904690 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904693 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904698 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904704 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904707 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904710 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:04:59.126083 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904713 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904716 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904720 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904723 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904728 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904731 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904734 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904737 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904740 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904743 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904745 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904748 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904751 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904754 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904757 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904760 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904762 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904765 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904768 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:04:59.126781 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904771 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904773 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904776 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904779 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904782 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904785 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904787 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904790 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904793 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904796 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904801 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904803 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904806 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904809 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904812 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904815 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904818 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904822 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904825 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904828 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:04:59.127402 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904830 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904833 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904836 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904839 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904841 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904844 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904848 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904852 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904855 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904858 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904861 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904864 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904867 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904870 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904873 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904876 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904878 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904881 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904884 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:04:59.128062 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904887 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904890 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904892 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904896 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904899 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904902 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904905 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904908 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904911 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904914 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904918 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904920 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904923 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904926 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904929 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904932 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904935 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904937 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904940 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:04:59.128698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:04:58.904943 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.904949 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.905089 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.908316 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.909563 2558 server.go:1019] "Starting client certificate rotation" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.909667 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.909726 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.912818 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.915118 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.924590 2558 log.go:25] "Validated CRI v1 runtime API" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.927348 2558 log.go:25] "Validated CRI v1 image API" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.928412 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 17:04:59.129322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.930929 2558 fs.go:135] Filesystem UUIDs: map[5603d632-9392-4fa7-a078-d33c91420fc9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 baa93d75-f811-4fa3-a122-b1ce2262ec64:/dev/nvme0n1p3] Jul 14 17:04:59.129744 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.930952 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.935570 2558 manager.go:216] Machine: {Timestamp:2026-07-14 17:04:58.93504085 +0000 UTC m=+1.504451631 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100062 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd7b8e6ae7cf063b62ab78ad0aa84 SystemUUID:ec2cd7b8-e6ae-7cf0-63b6-2ab78ad0aa84 BootID:757af1cb-5f00-4be4-a725-a19a93edf313 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1d:c0:af:58:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1d:c0:af:58:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:69:76:23:6e:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.935721 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.935817 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.938324 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.938369 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-240.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.939016 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.939028 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 17:04:59.129785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.939038 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.939787 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.941070 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.944264 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.944430 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.945094 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.945106 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.945118 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.945128 2558 kubelet.go:398] "Adding apiserver pod source" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.945138 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.945702 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.945727 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.951865 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954254 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954285 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954466 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954480 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954486 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954492 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954501 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.954506 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955115 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955123 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955130 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955136 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955146 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955154 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955788 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.955801 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.959970 2558 server.go:1268] "Started kubelet" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.960040 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.960555 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.960606 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.961687 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-240.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:58.961869 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:58.961899 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.962804 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 17:04:59.130108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.962808 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:58.966319 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6e381390e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:58.959935758 +0000 UTC m=+1.529346524,LastTimestamp:2026-07-14 17:04:58.959935758 +0000 UTC m=+1.529346524,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.968481 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.968507 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:58.968689 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.969526 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.969762 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.969777 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.969940 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.969950 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:58.970051 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.970297 2558 factory.go:55] Registering systemd factory Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.970315 2558 factory.go:223] Registration of the systemd container factory successfully Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.970553 2558 factory.go:153] Registering CRI-O factory Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.970582 2558 factory.go:223] Registration of the crio container factory successfully Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.970642 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.970663 2558 factory.go:103] Registering Raw factory Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.970676 2558 manager.go:1196] Started watching for new ooms in manager Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.971178 2558 manager.go:349] Starting recovery of all containers Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:58.975117 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:58.975169 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.984769 2558 manager.go:354] Recovery completed Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:58.989744 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:04:59.131403 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.069562 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.169761 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.169734 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.222653 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.222622 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:04:59.222790 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.222668 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:04:59.222790 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.222683 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:04:59.223394 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.223369 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 17:04:59.223394 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.223385 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 17:04:59.223394 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.223400 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:04:59.223614 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.223405 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:04:59.224120 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.224105 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:04:59.224213 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.224129 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:04:59.224213 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.224144 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:04:59.225504 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.225431 2558 policy_none.go:49] "None policy: Start" Jul 14 17:04:59.225504 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.225446 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 17:04:59.225504 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.225461 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 17:04:59.231895 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.231876 2558 policy_none.go:47] "Start" Jul 14 17:04:59.235355 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.235331 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Jul 14 17:04:59.235483 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.235397 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6f329ec9f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-240.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:59.222650015 +0000 UTC m=+1.792060787,LastTimestamp:2026-07-14 17:04:59.222650015 +0000 UTC m=+1.792060787,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.238544 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.238463 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6f32a4f55 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-240.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:59.222675285 +0000 UTC m=+1.792086052,LastTimestamp:2026-07-14 17:04:59.222675285 +0000 UTC m=+1.792086052,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.240486 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.240465 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:04:59.245886 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.245794 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6f32a8812 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-240.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:59.22268981 +0000 UTC m=+1.792100574,LastTimestamp:2026-07-14 17:04:59.22268981 +0000 UTC m=+1.792100574,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.250601 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.250385 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-240.ec2.internal.18c236d6f329ec9f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6f329ec9f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-240.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:59.222650015 +0000 UTC m=+1.792060787,LastTimestamp:2026-07-14 17:04:59.224116854 +0000 UTC m=+1.793527618,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.258673 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.258548 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-240.ec2.internal.18c236d6f32a4f55\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6f32a4f55 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-240.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:59.222675285 +0000 UTC m=+1.792086052,LastTimestamp:2026-07-14 17:04:59.224136809 +0000 UTC m=+1.793547579,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.270499 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.270476 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.270816 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.270732 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-240.ec2.internal.18c236d6f32a8812\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6f32a8812 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-240.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:59.22268981 +0000 UTC m=+1.792100574,LastTimestamp:2026-07-14 17:04:59.224149 +0000 UTC m=+1.793559765,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.280493 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.280454 2558 manager.go:337] "Starting Device Plugin manager" Jul 14 17:04:59.280614 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.280514 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 17:04:59.280614 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.280526 2558 server.go:87] "Starting device plugin registration server" Jul 14 17:04:59.280873 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.280858 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 17:04:59.280924 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.280876 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 17:04:59.281014 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.280980 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 17:04:59.281169 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.281145 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 17:04:59.281169 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.281162 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 17:04:59.281847 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.281827 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 17:04:59.281921 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.281883 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.293146 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.293059 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-240.ec2.internal.18c236d6f6c578ab default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-240.ec2.internal,UID:ip-10-0-128-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-240.ec2.internal,},FirstTimestamp:2026-07-14 17:04:59.283175595 +0000 UTC m=+1.852586345,LastTimestamp:2026-07-14 17:04:59.283175595 +0000 UTC m=+1.852586345,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-240.ec2.internal,}" Jul 14 17:04:59.311790 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.311756 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxmvq" Jul 14 17:04:59.345315 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.321210 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxmvq" Jul 14 17:04:59.373835 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.373800 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 17:04:59.375267 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.375243 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 17:04:59.375398 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.375287 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 17:04:59.375398 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.375311 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 17:04:59.375398 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.375358 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 17:04:59.381679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.381652 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:04:59.382457 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.382439 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:04:59.382518 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.382480 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:04:59.382518 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.382492 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:04:59.382518 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.382517 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.387037 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.387017 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:04:59.396056 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.396028 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.396169 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.396063 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-240.ec2.internal\": node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.421866 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.421832 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.476446 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.476372 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal"] Jul 14 17:04:59.476623 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.476487 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:04:59.477642 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.477598 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:04:59.477642 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.477638 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:04:59.477756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.477654 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:04:59.479632 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.479618 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:04:59.479750 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.479730 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.479793 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.479768 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:04:59.480644 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.480626 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:04:59.480717 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.480626 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:04:59.480717 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.480694 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:04:59.480717 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.480707 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:04:59.480814 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.480664 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:04:59.480814 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.480745 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:04:59.482387 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.482372 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.482459 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.482401 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:04:59.484640 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.484620 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:04:59.484752 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.484651 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:04:59.484752 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.484662 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:04:59.503694 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.503665 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-240.ec2.internal\" not found" node="ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.508736 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.508717 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-240.ec2.internal\" not found" node="ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.522702 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.522670 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.572603 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.572562 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d778c27593e0f865e91ef9b3785245bd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-240.ec2.internal\" (UID: \"d778c27593e0f865e91ef9b3785245bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.572678 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.572606 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/612add0975a0ebee44d9653e9e8597d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal\" (UID: \"612add0975a0ebee44d9653e9e8597d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.572678 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.572636 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/612add0975a0ebee44d9653e9e8597d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal\" (UID: \"612add0975a0ebee44d9653e9e8597d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.622984 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.622909 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.672822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.672786 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d778c27593e0f865e91ef9b3785245bd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-240.ec2.internal\" (UID: \"d778c27593e0f865e91ef9b3785245bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.672925 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.672826 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/612add0975a0ebee44d9653e9e8597d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal\" (UID: \"612add0975a0ebee44d9653e9e8597d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.672925 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.672854 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/612add0975a0ebee44d9653e9e8597d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal\" (UID: \"612add0975a0ebee44d9653e9e8597d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.672925 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.672900 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d778c27593e0f865e91ef9b3785245bd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-240.ec2.internal\" (UID: \"d778c27593e0f865e91ef9b3785245bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.673022 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.672925 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/612add0975a0ebee44d9653e9e8597d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal\" (UID: \"612add0975a0ebee44d9653e9e8597d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.673022 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.672957 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/612add0975a0ebee44d9653e9e8597d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal\" (UID: \"612add0975a0ebee44d9653e9e8597d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.723815 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.723753 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.807557 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.807498 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.811911 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.811880 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:04:59.823943 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.823908 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:04:59.910908 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.910808 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 17:04:59.911058 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:04:59.910982 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 17:04:59.924061 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:04:59.924017 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:05:00.025096 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:00.025037 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:05:00.067451 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.067418 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:00.125315 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:00.125270 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:05:00.227940 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:00.227862 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:05:00.324912 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.324862 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:00.326327 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.326302 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:00.326465 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.326347 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:00.326465 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.326363 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:00.328602 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:00.328581 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:05:00.332495 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.332465 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 16:59:59 +0000 UTC" deadline="2028-01-27 06:15:50.421951351 +0000 UTC" Jul 14 17:05:00.332586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.332496 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13477h10m50.089460617s" Jul 14 17:05:00.366682 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:00.366632 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612add0975a0ebee44d9653e9e8597d5.slice/crio-e00a45bcbed792eefab9971471161f388ce5d8b5c58d4b31f16e816312d81838 WatchSource:0}: Error finding container e00a45bcbed792eefab9971471161f388ce5d8b5c58d4b31f16e816312d81838: Status 404 returned error can't find the container with id e00a45bcbed792eefab9971471161f388ce5d8b5c58d4b31f16e816312d81838 Jul 14 17:05:00.375247 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.375230 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:05:00.381583 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.381521 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" event={"ID":"612add0975a0ebee44d9653e9e8597d5","Type":"ContainerStarted","Data":"e00a45bcbed792eefab9971471161f388ce5d8b5c58d4b31f16e816312d81838"} Jul 14 17:05:00.384074 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:00.384047 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd778c27593e0f865e91ef9b3785245bd.slice/crio-d32b4909a62a15adf1288f8049b84a60327b35145e90e978c952e6d8ee8b1924 WatchSource:0}: Error finding container d32b4909a62a15adf1288f8049b84a60327b35145e90e978c952e6d8ee8b1924: Status 404 returned error can't find the container with id d32b4909a62a15adf1288f8049b84a60327b35145e90e978c952e6d8ee8b1924 Jul 14 17:05:00.413274 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.413241 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:00.429073 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:00.429017 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-240.ec2.internal\" not found" Jul 14 17:05:00.507456 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.507385 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:00.571523 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.571484 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" Jul 14 17:05:00.585654 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.585629 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:05:00.586656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.586639 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" Jul 14 17:05:00.602910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.602882 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:05:00.947903 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.947871 2558 apiserver.go:52] "Watching apiserver" Jul 14 17:05:00.957291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.956204 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 17:05:00.957291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.956843 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal","openshift-multus/multus-additional-cni-plugins-vr2g8","openshift-network-diagnostics/network-check-target-47glh","openshift-ovn-kubernetes/ovnkube-node-6sjrd","kube-system/global-pull-secret-syncer-s29kb","kube-system/konnectivity-agent-wckws","openshift-cluster-node-tuning-operator/tuned-c89hx","openshift-dns/node-resolver-pkfmh","openshift-image-registry/node-ca-2fd85","openshift-multus/multus-fscf5","openshift-multus/network-metrics-daemon-sr7b5","openshift-network-operator/iptables-alerter-zq4sn","kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n"] Jul 14 17:05:00.959172 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.959145 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.961781 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.961756 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.962687 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.962406 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 17:05:00.962687 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.962502 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 17:05:00.962687 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.962543 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5b8xk\"" Jul 14 17:05:00.962940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.962802 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 17:05:00.963078 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.963026 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 17:05:00.975143 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.971491 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 17:05:00.975143 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.972659 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 17:05:00.975143 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.972871 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.975143 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.973670 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cdmfg\"" Jul 14 17:05:00.975143 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.974086 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 17:05:00.977885 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.977828 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-etc-kubernetes\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.977885 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.977874 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-socket-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.978107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.977904 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-system-cni-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.977933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-cni-bin\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.977965 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-system-cni-dir\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.978107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.977996 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-registration-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.978107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978024 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-device-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.978107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e7be499-7243-4991-b4a7-57b1f1070a94-cni-binary-copy\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978091 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-k8s-cni-cncf-io\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978123 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-netns\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978160 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-kubelet\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978194 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d9dv\" (UniqueName: \"kubernetes.io/projected/5e7be499-7243-4991-b4a7-57b1f1070a94-kube-api-access-6d9dv\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978264 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.978479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978297 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc646b1-d627-4d8f-a261-d84275abdb68-tmp-dir\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:00.978479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978411 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-os-release\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978439 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-os-release\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978510 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978555 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-etc-selinux\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978592 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-cnibin\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978649 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-socket-dir-parent\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978682 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-conf-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978712 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cni-binary-copy\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978750 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-cni-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978784 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-cni-multus\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978820 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-multus-certs\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.978870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5zmg\" (UniqueName: \"kubernetes.io/projected/2bc646b1-d627-4d8f-a261-d84275abdb68-kube-api-access-s5zmg\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978885 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-daemon-config\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978920 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978957 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.978993 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq5xh\" (UniqueName: \"kubernetes.io/projected/229304ae-1b47-4f12-b7a5-8e7ce52b0637-kube-api-access-gq5xh\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.979022 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cnibin\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.979063 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-sys-fs\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.979097 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r66nc\" (UniqueName: \"kubernetes.io/projected/750be559-171d-4c27-b164-61129e060388-kube-api-access-r66nc\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.979130 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bc646b1-d627-4d8f-a261-d84275abdb68-hosts-file\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:00.979384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.979185 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-hostroot\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:00.982984 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.981303 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:00.985546 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.985067 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xs6gm\"" Jul 14 17:05:00.985546 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.985178 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 17:05:00.985546 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.985250 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jvm84\"" Jul 14 17:05:00.985546 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.985358 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 17:05:00.985546 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.985436 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 17:05:00.986446 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.986158 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 17:05:00.986446 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.986179 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 17:05:00.987425 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.986929 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:00.990113 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.989638 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:00.990113 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.989673 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gdjp2\"" Jul 14 17:05:00.990113 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.989739 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:00.990322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.990192 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:00.991671 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.991649 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:00.991794 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.991760 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:05:00.993252 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.993228 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 17:05:00.993482 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.993465 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bvg2x\"" Jul 14 17:05:00.993610 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.993589 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 17:05:00.993786 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.993761 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:00.993880 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:00.993852 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:00.994352 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.994330 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 17:05:00.995371 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.994729 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 17:05:00.995371 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.994948 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lsrh4\"" Jul 14 17:05:00.995371 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.995146 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 17:05:00.995753 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.995720 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:00.995814 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:00.995793 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:00.997987 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:00.997963 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.001178 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.001147 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:01.001291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.001230 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 17:05:01.001291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.001239 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:01.002121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.001478 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.002121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.001504 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:01.002305 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.002175 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2zr5h\"" Jul 14 17:05:01.002305 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.002191 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:01.002305 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.002258 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:01.004724 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.004628 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 17:05:01.004838 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.004807 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 17:05:01.005101 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.004699 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 17:05:01.005101 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.004706 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 17:05:01.005473 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.005456 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 17:05:01.006277 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.006260 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xjg44\"" Jul 14 17:05:01.006684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.006558 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 17:05:01.029406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.029373 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5vxvr" Jul 14 17:05:01.036963 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.036883 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5vxvr" Jul 14 17:05:01.071687 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.071658 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 17:05:01.079903 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.079867 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-socket-dir-parent\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.080088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.079919 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6992d303-6bd8-485d-b532-f782c6809cbc-host\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.080088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.079948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-modprobe-d\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.080088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.079978 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-log-socket\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.080088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080007 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.080088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080038 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-cni-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.080088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5zmg\" (UniqueName: \"kubernetes.io/projected/2bc646b1-d627-4d8f-a261-d84275abdb68-kube-api-access-s5zmg\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:01.080088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080073 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-socket-dir-parent\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.080481 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080093 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-node-log\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.080481 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080138 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovnkube-script-lib\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.080481 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080184 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.080481 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080297 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-cni-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.080481 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080343 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq5xh\" (UniqueName: \"kubernetes.io/projected/229304ae-1b47-4f12-b7a5-8e7ce52b0637-kube-api-access-gq5xh\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.080481 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080387 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/555eafcf-ccb1-4454-b367-f5b038305022-agent-certs\") pod \"konnectivity-agent-wckws\" (UID: \"555eafcf-ccb1-4454-b367-f5b038305022\") " pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:01.080481 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080415 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-systemd\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.081969 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.081915 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.082084 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.080463 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-ovn\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.082084 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082070 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovnkube-config\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.082171 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082134 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cnibin\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.082204 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082179 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r66nc\" (UniqueName: \"kubernetes.io/projected/750be559-171d-4c27-b164-61129e060388-kube-api-access-r66nc\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.082335 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysctl-d\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.082335 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c63b8442-55f0-4748-bd35-9902b296da10-etc-tuned\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.082335 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082334 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-kubelet\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.082589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082380 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cnibin\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.082589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-etc-kubernetes\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.082589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082508 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7268fe4b-985a-40f4-9eff-317b73bab0cd-iptables-alerter-script\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.082589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082583 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-host\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.082825 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082617 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c63b8442-55f0-4748-bd35-9902b296da10-tmp\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.082825 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082650 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsdch\" (UniqueName: \"kubernetes.io/projected/c63b8442-55f0-4748-bd35-9902b296da10-kube-api-access-nsdch\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.082825 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082681 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-cni-bin\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.082825 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082736 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szzp5\" (UniqueName: \"kubernetes.io/projected/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-kube-api-access-szzp5\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.082825 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082764 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-etc-kubernetes\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.082825 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082772 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-system-cni-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082826 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-system-cni-dir\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082834 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-system-cni-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082876 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-registration-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082930 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8vkv\" (UniqueName: \"kubernetes.io/projected/6992d303-6bd8-485d-b532-f782c6809cbc-kube-api-access-c8vkv\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.082965 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083008 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-lib-modules\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083039 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-system-cni-dir\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083048 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-systemd-units\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-systemd\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.083123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083094 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-registration-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083131 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e7be499-7243-4991-b4a7-57b1f1070a94-cni-binary-copy\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083182 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-k8s-cni-cncf-io\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083309 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-netns\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083339 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083403 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-run\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083435 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-sys\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083466 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba8203bc-44c2-4091-ae5d-f0fb93aba210-kubelet-config\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083499 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-os-release\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083558 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-etc-selinux\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083620 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysconfig\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.083651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083655 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-run-ovn-kubernetes\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083692 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-cnibin\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083724 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-conf-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083755 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cni-binary-copy\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083821 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovn-node-metrics-cert\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-cni-multus\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083882 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e7be499-7243-4991-b4a7-57b1f1070a94-cni-binary-copy\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-multus-certs\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.083990 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-kubernetes\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084021 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba8203bc-44c2-4091-ae5d-f0fb93aba210-dbus\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084047 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-slash\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084077 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-etc-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-daemon-config\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084146 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.084220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084177 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dclf6\" (UniqueName: \"kubernetes.io/projected/7268fe4b-985a-40f4-9eff-317b73bab0cd-kube-api-access-dclf6\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084202 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-run-netns\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084236 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-sys-fs\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084273 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bc646b1-d627-4d8f-a261-d84275abdb68-hosts-file\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084303 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6992d303-6bd8-485d-b532-f782c6809cbc-serviceca\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084465 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cni-binary-copy\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084499 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-cni-netd\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084529 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-hostroot\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084533 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-conf-dir\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084564 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-socket-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084579 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-cni-multus\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-cni-bin\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084646 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-device-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.084976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084675 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084678 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-os-release\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084711 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-var-lib-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-kubelet\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084778 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d9dv\" (UniqueName: \"kubernetes.io/projected/5e7be499-7243-4991-b4a7-57b1f1070a94-kube-api-access-6d9dv\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084811 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc646b1-d627-4d8f-a261-d84275abdb68-tmp-dir\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084838 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/555eafcf-ccb1-4454-b367-f5b038305022-konnectivity-ca\") pod \"konnectivity-agent-wckws\" (UID: \"555eafcf-ccb1-4454-b367-f5b038305022\") " pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084871 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbm9t\" (UniqueName: \"kubernetes.io/projected/86199c5a-74c9-446b-9aa5-028fb5224e6c-kube-api-access-sbm9t\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084908 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7268fe4b-985a-40f4-9eff-317b73bab0cd-host-slash\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084941 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysctl-conf\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.084972 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-var-lib-kubelet\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085001 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-os-release\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085035 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085067 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.085680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085099 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-env-overrides\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085172 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/229304ae-1b47-4f12-b7a5-8e7ce52b0637-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085212 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-multus-certs\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085266 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-cni-bin\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085314 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-socket-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085325 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-device-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085399 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-var-lib-kubelet\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085647 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc646b1-d627-4d8f-a261-d84275abdb68-tmp-dir\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085762 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-k8s-cni-cncf-io\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085766 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bc646b1-d627-4d8f-a261-d84275abdb68-hosts-file\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085832 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-hostroot\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085860 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-sys-fs\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085975 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-host-run-netns\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.085985 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-os-release\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.086093 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.086406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.086105 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/750be559-171d-4c27-b164-61129e060388-etc-selinux\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.087218 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.086119 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e7be499-7243-4991-b4a7-57b1f1070a94-cnibin\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.087218 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.086274 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/229304ae-1b47-4f12-b7a5-8e7ce52b0637-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.087218 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.086644 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e7be499-7243-4991-b4a7-57b1f1070a94-multus-daemon-config\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.092937 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.092895 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 17:05:01.099309 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.099203 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq5xh\" (UniqueName: \"kubernetes.io/projected/229304ae-1b47-4f12-b7a5-8e7ce52b0637-kube-api-access-gq5xh\") pod \"multus-additional-cni-plugins-vr2g8\" (UID: \"229304ae-1b47-4f12-b7a5-8e7ce52b0637\") " pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.100059 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.099994 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d9dv\" (UniqueName: \"kubernetes.io/projected/5e7be499-7243-4991-b4a7-57b1f1070a94-kube-api-access-6d9dv\") pod \"multus-fscf5\" (UID: \"5e7be499-7243-4991-b4a7-57b1f1070a94\") " pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.100652 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.100594 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5zmg\" (UniqueName: \"kubernetes.io/projected/2bc646b1-d627-4d8f-a261-d84275abdb68-kube-api-access-s5zmg\") pod \"node-resolver-pkfmh\" (UID: \"2bc646b1-d627-4d8f-a261-d84275abdb68\") " pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:01.103155 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.103110 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r66nc\" (UniqueName: \"kubernetes.io/projected/750be559-171d-4c27-b164-61129e060388-kube-api-access-r66nc\") pod \"aws-ebs-csi-driver-node-m5t6n\" (UID: \"750be559-171d-4c27-b164-61129e060388\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.185897 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.185853 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/555eafcf-ccb1-4454-b367-f5b038305022-agent-certs\") pod \"konnectivity-agent-wckws\" (UID: \"555eafcf-ccb1-4454-b367-f5b038305022\") " pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:01.185897 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.185902 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-systemd\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.185996 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-systemd\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186035 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-ovn\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186060 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovnkube-config\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186100 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysctl-d\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186125 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c63b8442-55f0-4748-bd35-9902b296da10-etc-tuned\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186149 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-kubelet\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186153 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-ovn\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186176 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7268fe4b-985a-40f4-9eff-317b73bab0cd-iptables-alerter-script\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-host\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186244 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c63b8442-55f0-4748-bd35-9902b296da10-tmp\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsdch\" (UniqueName: \"kubernetes.io/projected/c63b8442-55f0-4748-bd35-9902b296da10-kube-api-access-nsdch\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186302 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-cni-bin\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186327 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szzp5\" (UniqueName: \"kubernetes.io/projected/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-kube-api-access-szzp5\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186332 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysctl-d\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186357 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8vkv\" (UniqueName: \"kubernetes.io/projected/6992d303-6bd8-485d-b532-f782c6809cbc-kube-api-access-c8vkv\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186404 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186454 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-lib-modules\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-systemd-units\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-systemd\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186542 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-run\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186565 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-sys\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186608 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba8203bc-44c2-4091-ae5d-f0fb93aba210-kubelet-config\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.186735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186641 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysconfig\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186660 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-cni-bin\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186667 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-run-ovn-kubernetes\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186725 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-host\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovn-node-metrics-cert\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186759 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba8203bc-44c2-4091-ae5d-f0fb93aba210-kubelet-config\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186772 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-kubernetes\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186775 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7268fe4b-985a-40f4-9eff-317b73bab0cd-iptables-alerter-script\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186790 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-run-ovn-kubernetes\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186798 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba8203bc-44c2-4091-ae5d-f0fb93aba210-dbus\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186828 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysconfig\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186870 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-slash\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186986 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-lib-modules\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187026 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-systemd-units\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187075 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-run\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.187576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187119 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-sys\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.186832 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-slash\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187179 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-etc-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187213 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dclf6\" (UniqueName: \"kubernetes.io/projected/7268fe4b-985a-40f4-9eff-317b73bab0cd-kube-api-access-dclf6\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187225 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-kubernetes\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-run-netns\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187177 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-kubelet\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6992d303-6bd8-485d-b532-f782c6809cbc-serviceca\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187290 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-etc-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.187296 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187311 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187332 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-run-netns\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187338 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-cni-netd\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.187369 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs podName:86199c5a-74c9-446b-9aa5-028fb5224e6c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:01.687345484 +0000 UTC m=+4.256756258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs") pod "network-metrics-daemon-sr7b5" (UID: "86199c5a-74c9-446b-9aa5-028fb5224e6c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:01.188383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187375 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-cni-netd\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-var-lib-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/555eafcf-ccb1-4454-b367-f5b038305022-konnectivity-ca\") pod \"konnectivity-agent-wckws\" (UID: \"555eafcf-ccb1-4454-b367-f5b038305022\") " pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbm9t\" (UniqueName: \"kubernetes.io/projected/86199c5a-74c9-446b-9aa5-028fb5224e6c-kube-api-access-sbm9t\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.187544 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187555 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7268fe4b-985a-40f4-9eff-317b73bab0cd-host-slash\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187582 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysctl-conf\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.187621 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret podName:ba8203bc-44c2-4091-ae5d-f0fb93aba210 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:01.68759712 +0000 UTC m=+4.257007871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret") pod "global-pull-secret-syncer-s29kb" (UID: "ba8203bc-44c2-4091-ae5d-f0fb93aba210") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-var-lib-kubelet\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187681 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-env-overrides\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187736 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6992d303-6bd8-485d-b532-f782c6809cbc-host\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187762 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-modprobe-d\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.189141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187788 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-log-socket\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187818 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187853 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-node-log\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187881 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovnkube-script-lib\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.187934 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-sysctl-conf\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188004 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7268fe4b-985a-40f4-9eff-317b73bab0cd-host-slash\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188006 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-var-lib-kubelet\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188075 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-log-socket\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188114 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c63b8442-55f0-4748-bd35-9902b296da10-etc-modprobe-d\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188124 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188162 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188171 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-node-log\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188251 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/555eafcf-ccb1-4454-b367-f5b038305022-konnectivity-ca\") pod \"konnectivity-agent-wckws\" (UID: \"555eafcf-ccb1-4454-b367-f5b038305022\") " pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188305 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-var-lib-openvswitch\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188344 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6992d303-6bd8-485d-b532-f782c6809cbc-host\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.190004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188490 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovnkube-script-lib\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188506 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba8203bc-44c2-4091-ae5d-f0fb93aba210-dbus\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.190845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188585 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-env-overrides\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-run-systemd\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188833 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovnkube-config\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.190845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.188912 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6992d303-6bd8-485d-b532-f782c6809cbc-serviceca\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.190845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.189773 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c63b8442-55f0-4748-bd35-9902b296da10-etc-tuned\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.190845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.190391 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/555eafcf-ccb1-4454-b367-f5b038305022-agent-certs\") pod \"konnectivity-agent-wckws\" (UID: \"555eafcf-ccb1-4454-b367-f5b038305022\") " pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:01.191200 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.191171 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-ovn-node-metrics-cert\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.191651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.191584 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c63b8442-55f0-4748-bd35-9902b296da10-tmp\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.197598 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.197566 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:01.197774 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.197613 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:01.197774 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.197631 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5dm5h for pod openshift-network-diagnostics/network-check-target-47glh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:01.197774 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.197715 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h podName:3300e5a3-e457-4254-81a3-76412fdb24eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:01.697690976 +0000 UTC m=+4.267101750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5dm5h" (UniqueName: "kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h") pod "network-check-target-47glh" (UID: "3300e5a3-e457-4254-81a3-76412fdb24eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:01.198943 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.198854 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsdch\" (UniqueName: \"kubernetes.io/projected/c63b8442-55f0-4748-bd35-9902b296da10-kube-api-access-nsdch\") pod \"tuned-c89hx\" (UID: \"c63b8442-55f0-4748-bd35-9902b296da10\") " pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.199707 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.199652 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8vkv\" (UniqueName: \"kubernetes.io/projected/6992d303-6bd8-485d-b532-f782c6809cbc-kube-api-access-c8vkv\") pod \"node-ca-2fd85\" (UID: \"6992d303-6bd8-485d-b532-f782c6809cbc\") " pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.200118 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.200093 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szzp5\" (UniqueName: \"kubernetes.io/projected/3ca33e99-b10d-406f-b4fa-3be9bcbd7467-kube-api-access-szzp5\") pod \"ovnkube-node-6sjrd\" (UID: \"3ca33e99-b10d-406f-b4fa-3be9bcbd7467\") " pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.204315 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.204281 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dclf6\" (UniqueName: \"kubernetes.io/projected/7268fe4b-985a-40f4-9eff-317b73bab0cd-kube-api-access-dclf6\") pod \"iptables-alerter-zq4sn\" (UID: \"7268fe4b-985a-40f4-9eff-317b73bab0cd\") " pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.204652 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.204538 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbm9t\" (UniqueName: \"kubernetes.io/projected/86199c5a-74c9-446b-9aa5-028fb5224e6c-kube-api-access-sbm9t\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:01.289113 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.288804 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fscf5" Jul 14 17:05:01.301576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.301535 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" Jul 14 17:05:01.304479 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.304358 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7be499_7243_4991_b4a7_57b1f1070a94.slice/crio-67676a2f89e4f0c2d9dbdfdaacf1916b773f8d0624181c4010772170f2083d1c WatchSource:0}: Error finding container 67676a2f89e4f0c2d9dbdfdaacf1916b773f8d0624181c4010772170f2083d1c: Status 404 returned error can't find the container with id 67676a2f89e4f0c2d9dbdfdaacf1916b773f8d0624181c4010772170f2083d1c Jul 14 17:05:01.311410 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.311170 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" Jul 14 17:05:01.316464 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.316428 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229304ae_1b47_4f12_b7a5_8e7ce52b0637.slice/crio-9b9e61bd8a02916ee4ed1421c478fb84be6477fa181caa54c2a9ea7826c9fdaa WatchSource:0}: Error finding container 9b9e61bd8a02916ee4ed1421c478fb84be6477fa181caa54c2a9ea7826c9fdaa: Status 404 returned error can't find the container with id 9b9e61bd8a02916ee4ed1421c478fb84be6477fa181caa54c2a9ea7826c9fdaa Jul 14 17:05:01.320845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.320723 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pkfmh" Jul 14 17:05:01.323572 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.323539 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750be559_171d_4c27_b164_61129e060388.slice/crio-efada4d173b4af19e61659e6c7904151df67d4275366484d4454fd89d25d8dc6 WatchSource:0}: Error finding container efada4d173b4af19e61659e6c7904151df67d4275366484d4454fd89d25d8dc6: Status 404 returned error can't find the container with id efada4d173b4af19e61659e6c7904151df67d4275366484d4454fd89d25d8dc6 Jul 14 17:05:01.330763 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.330728 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c89hx" Jul 14 17:05:01.336316 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.336277 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc646b1_d627_4d8f_a261_d84275abdb68.slice/crio-ed6ef2213762a07d24db00c3dfe96dfe3c61de1650b8965fcde1cb79290481db WatchSource:0}: Error finding container ed6ef2213762a07d24db00c3dfe96dfe3c61de1650b8965fcde1cb79290481db: Status 404 returned error can't find the container with id ed6ef2213762a07d24db00c3dfe96dfe3c61de1650b8965fcde1cb79290481db Jul 14 17:05:01.340928 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.340693 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:01.348641 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.348612 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2fd85" Jul 14 17:05:01.350103 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.350061 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63b8442_55f0_4748_bd35_9902b296da10.slice/crio-5a4eb1394396d29c6e16a9f6c6b1456ae4735a9f956d6e535197f72dd6b47580 WatchSource:0}: Error finding container 5a4eb1394396d29c6e16a9f6c6b1456ae4735a9f956d6e535197f72dd6b47580: Status 404 returned error can't find the container with id 5a4eb1394396d29c6e16a9f6c6b1456ae4735a9f956d6e535197f72dd6b47580 Jul 14 17:05:01.354532 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.354404 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555eafcf_ccb1_4454_b367_f5b038305022.slice/crio-e88888b1aaa0b95ad9d4479c11c6f2cde76d961646fac145c272dffe7d37e754 WatchSource:0}: Error finding container e88888b1aaa0b95ad9d4479c11c6f2cde76d961646fac145c272dffe7d37e754: Status 404 returned error can't find the container with id e88888b1aaa0b95ad9d4479c11c6f2cde76d961646fac145c272dffe7d37e754 Jul 14 17:05:01.362482 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.362220 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zq4sn" Jul 14 17:05:01.364946 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.364770 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6992d303_6bd8_485d_b532_f782c6809cbc.slice/crio-11b3f76158059681ba23909df1f253555a7837ed929c6e0ceccb8dc00f9bf346 WatchSource:0}: Error finding container 11b3f76158059681ba23909df1f253555a7837ed929c6e0ceccb8dc00f9bf346: Status 404 returned error can't find the container with id 11b3f76158059681ba23909df1f253555a7837ed929c6e0ceccb8dc00f9bf346 Jul 14 17:05:01.370733 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.369972 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:01.376625 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.376584 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7268fe4b_985a_40f4_9eff_317b73bab0cd.slice/crio-143a20d91ce2958728bfdeeb030a99717415d8ec740c69d363f19e901ac7dd6c WatchSource:0}: Error finding container 143a20d91ce2958728bfdeeb030a99717415d8ec740c69d363f19e901ac7dd6c: Status 404 returned error can't find the container with id 143a20d91ce2958728bfdeeb030a99717415d8ec740c69d363f19e901ac7dd6c Jul 14 17:05:01.385692 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:01.385632 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca33e99_b10d_406f_b4fa_3be9bcbd7467.slice/crio-c13ea265d9f5e59d19a23f741792bd1709df651cc3d9bb23e0b33e5621b71ea0 WatchSource:0}: Error finding container c13ea265d9f5e59d19a23f741792bd1709df651cc3d9bb23e0b33e5621b71ea0: Status 404 returned error can't find the container with id c13ea265d9f5e59d19a23f741792bd1709df651cc3d9bb23e0b33e5621b71ea0 Jul 14 17:05:01.387909 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.387867 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wckws" event={"ID":"555eafcf-ccb1-4454-b367-f5b038305022","Type":"ContainerStarted","Data":"e88888b1aaa0b95ad9d4479c11c6f2cde76d961646fac145c272dffe7d37e754"} Jul 14 17:05:01.391063 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.390927 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pkfmh" event={"ID":"2bc646b1-d627-4d8f-a261-d84275abdb68","Type":"ContainerStarted","Data":"ed6ef2213762a07d24db00c3dfe96dfe3c61de1650b8965fcde1cb79290481db"} Jul 14 17:05:01.393073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.392987 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerStarted","Data":"9b9e61bd8a02916ee4ed1421c478fb84be6477fa181caa54c2a9ea7826c9fdaa"} Jul 14 17:05:01.396290 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.396259 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fscf5" event={"ID":"5e7be499-7243-4991-b4a7-57b1f1070a94","Type":"ContainerStarted","Data":"67676a2f89e4f0c2d9dbdfdaacf1916b773f8d0624181c4010772170f2083d1c"} Jul 14 17:05:01.399923 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.399208 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zq4sn" event={"ID":"7268fe4b-985a-40f4-9eff-317b73bab0cd","Type":"ContainerStarted","Data":"143a20d91ce2958728bfdeeb030a99717415d8ec740c69d363f19e901ac7dd6c"} Jul 14 17:05:01.403474 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.402306 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c89hx" event={"ID":"c63b8442-55f0-4748-bd35-9902b296da10","Type":"ContainerStarted","Data":"5a4eb1394396d29c6e16a9f6c6b1456ae4735a9f956d6e535197f72dd6b47580"} Jul 14 17:05:01.406321 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.406225 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" event={"ID":"750be559-171d-4c27-b164-61129e060388","Type":"ContainerStarted","Data":"efada4d173b4af19e61659e6c7904151df67d4275366484d4454fd89d25d8dc6"} Jul 14 17:05:01.408871 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.408838 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" event={"ID":"d778c27593e0f865e91ef9b3785245bd","Type":"ContainerStarted","Data":"d32b4909a62a15adf1288f8049b84a60327b35145e90e978c952e6d8ee8b1924"} Jul 14 17:05:01.410911 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.410874 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2fd85" event={"ID":"6992d303-6bd8-485d-b532-f782c6809cbc","Type":"ContainerStarted","Data":"11b3f76158059681ba23909df1f253555a7837ed929c6e0ceccb8dc00f9bf346"} Jul 14 17:05:01.691890 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.691844 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:01.692087 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.691898 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:01.692087 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.692009 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:01.692087 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.692066 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret podName:ba8203bc-44c2-4091-ae5d-f0fb93aba210 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:02.692052772 +0000 UTC m=+5.261463522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret") pod "global-pull-secret-syncer-s29kb" (UID: "ba8203bc-44c2-4091-ae5d-f0fb93aba210") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:01.692242 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.692009 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:01.692242 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.692140 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs podName:86199c5a-74c9-446b-9aa5-028fb5224e6c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:02.692126271 +0000 UTC m=+5.261537035 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs") pod "network-metrics-daemon-sr7b5" (UID: "86199c5a-74c9-446b-9aa5-028fb5224e6c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:01.792918 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:01.792880 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:01.793105 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.793047 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:01.793105 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.793074 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:01.793105 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.793089 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5dm5h for pod openshift-network-diagnostics/network-check-target-47glh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:01.793282 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:01.793163 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h podName:3300e5a3-e457-4254-81a3-76412fdb24eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:02.793140076 +0000 UTC m=+5.362550830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dm5h" (UniqueName: "kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h") pod "network-check-target-47glh" (UID: "3300e5a3-e457-4254-81a3-76412fdb24eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:02.039636 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.039562 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:00:01 +0000 UTC" deadline="2028-02-20 17:50:54.650034866 +0000 UTC" Jul 14 17:05:02.039636 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.039630 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14064h45m52.610409784s" Jul 14 17:05:02.376082 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.375523 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:02.376082 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.375682 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:02.376346 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.376119 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:02.376346 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.376154 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:02.376346 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.376236 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:02.376346 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.376324 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:02.414733 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.414593 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"c13ea265d9f5e59d19a23f741792bd1709df651cc3d9bb23e0b33e5621b71ea0"} Jul 14 17:05:02.700709 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.700662 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:02.700869 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.700735 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:02.700930 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.700898 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:02.700987 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.700967 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret podName:ba8203bc-44c2-4091-ae5d-f0fb93aba210 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:04.700943339 +0000 UTC m=+7.270354105 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret") pod "global-pull-secret-syncer-s29kb" (UID: "ba8203bc-44c2-4091-ae5d-f0fb93aba210") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:02.701163 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.701144 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:02.701248 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.701204 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs podName:86199c5a-74c9-446b-9aa5-028fb5224e6c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:04.701186403 +0000 UTC m=+7.270597165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs") pod "network-metrics-daemon-sr7b5" (UID: "86199c5a-74c9-446b-9aa5-028fb5224e6c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:02.803455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:02.803412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:02.803692 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.803670 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:02.803766 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.803705 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:02.803766 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.803722 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5dm5h for pod openshift-network-diagnostics/network-check-target-47glh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:02.803928 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:02.803794 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h podName:3300e5a3-e457-4254-81a3-76412fdb24eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:04.803771618 +0000 UTC m=+7.373182388 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dm5h" (UniqueName: "kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h") pod "network-check-target-47glh" (UID: "3300e5a3-e457-4254-81a3-76412fdb24eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:03.040847 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:03.040518 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:00:01 +0000 UTC" deadline="2028-01-16 23:24:22.608916304 +0000 UTC" Jul 14 17:05:03.040847 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:03.040849 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13230h19m19.568075974s" Jul 14 17:05:03.422910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:03.422867 2558 generic.go:356] "Generic (PLEG): container finished" podID="612add0975a0ebee44d9653e9e8597d5" containerID="cf037f53687eb8e34efc17c59f2d468a3d4e4f565d6649b74ec36712b52cbbdf" exitCode=0 Jul 14 17:05:03.423110 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:03.422999 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" event={"ID":"612add0975a0ebee44d9653e9e8597d5","Type":"ContainerDied","Data":"cf037f53687eb8e34efc17c59f2d468a3d4e4f565d6649b74ec36712b52cbbdf"} Jul 14 17:05:03.430664 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:03.430621 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" event={"ID":"d778c27593e0f865e91ef9b3785245bd","Type":"ContainerStarted","Data":"2d7468d9b4c4788fe8308306600b6344da9dc5d01d926ae70845e80b27553437"} Jul 14 17:05:04.376289 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.376253 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:04.376796 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.376392 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:04.376857 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.376824 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:04.376968 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.376944 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:04.377044 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.377028 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:04.377132 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.377114 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:04.442753 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.442715 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" event={"ID":"612add0975a0ebee44d9653e9e8597d5","Type":"ContainerStarted","Data":"6ab86a5c1eb1b7a253433ca0c23e3703744ce6e4126584576264439304f87f70"} Jul 14 17:05:04.477484 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.476829 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-240.ec2.internal" podStartSLOduration=4.476794791 podStartE2EDuration="4.476794791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:03.464286466 +0000 UTC m=+6.033697258" watchObservedRunningTime="2026-07-14 17:05:04.476794791 +0000 UTC m=+7.046205565" Jul 14 17:05:04.477484 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.477037 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-240.ec2.internal" podStartSLOduration=4.477029106 podStartE2EDuration="4.477029106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:04.475664827 +0000 UTC m=+7.045075621" watchObservedRunningTime="2026-07-14 17:05:04.477029106 +0000 UTC m=+7.046439878" Jul 14 17:05:04.722096 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.720993 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:04.722096 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.721072 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:04.722096 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.721224 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:04.722096 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.721290 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret podName:ba8203bc-44c2-4091-ae5d-f0fb93aba210 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:08.721272655 +0000 UTC m=+11.290683407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret") pod "global-pull-secret-syncer-s29kb" (UID: "ba8203bc-44c2-4091-ae5d-f0fb93aba210") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:04.722096 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.721882 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:04.722096 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.721967 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs podName:86199c5a-74c9-446b-9aa5-028fb5224e6c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:08.721921023 +0000 UTC m=+11.291331774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs") pod "network-metrics-daemon-sr7b5" (UID: "86199c5a-74c9-446b-9aa5-028fb5224e6c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:04.822511 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:04.822466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:04.822727 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.822713 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:04.822817 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.822734 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:04.822817 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.822748 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5dm5h for pod openshift-network-diagnostics/network-check-target-47glh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:04.822817 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:04.822810 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h podName:3300e5a3-e457-4254-81a3-76412fdb24eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:08.822789895 +0000 UTC m=+11.392200658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dm5h" (UniqueName: "kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h") pod "network-check-target-47glh" (UID: "3300e5a3-e457-4254-81a3-76412fdb24eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:06.376335 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:06.376299 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:06.376335 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:06.376334 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:06.377043 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:06.376309 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:06.377043 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:06.376435 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:06.377043 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:06.376572 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:06.377043 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:06.376659 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:08.376441 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:08.376375 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:08.376934 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:08.376515 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:08.376934 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.376524 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:08.376934 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:08.376568 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:08.376934 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.376682 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:08.376934 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.376777 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:08.757762 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:08.757668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:08.757948 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:08.757765 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:08.758009 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.757943 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:08.758009 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.757966 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:08.758009 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.757998 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret podName:ba8203bc-44c2-4091-ae5d-f0fb93aba210 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:16.757983634 +0000 UTC m=+19.327394386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret") pod "global-pull-secret-syncer-s29kb" (UID: "ba8203bc-44c2-4091-ae5d-f0fb93aba210") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:08.758160 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.758023 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs podName:86199c5a-74c9-446b-9aa5-028fb5224e6c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:16.758004701 +0000 UTC m=+19.327415467 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs") pod "network-metrics-daemon-sr7b5" (UID: "86199c5a-74c9-446b-9aa5-028fb5224e6c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:08.858206 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:08.858170 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:08.858400 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.858378 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:08.858400 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.858398 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:08.858503 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.858412 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5dm5h for pod openshift-network-diagnostics/network-check-target-47glh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:08.858503 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:08.858479 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h podName:3300e5a3-e457-4254-81a3-76412fdb24eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:16.858457807 +0000 UTC m=+19.427868559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dm5h" (UniqueName: "kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h") pod "network-check-target-47glh" (UID: "3300e5a3-e457-4254-81a3-76412fdb24eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:10.376559 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:10.376516 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:10.377032 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:10.376516 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:10.377032 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:10.376516 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:10.377032 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:10.376686 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:10.377032 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:10.376794 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:10.377032 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:10.376890 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:12.376554 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:12.376507 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:12.376554 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:12.376526 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:12.377004 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:12.376506 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:12.377004 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:12.376650 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:12.377004 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:12.376703 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:12.377004 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:12.376780 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:14.376325 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:14.376288 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:14.376743 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:14.376332 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:14.376743 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:14.376332 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:14.376743 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:14.376405 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:14.376743 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:14.376513 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:14.376743 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:14.376632 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:16.376262 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:16.376219 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:16.376262 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:16.376244 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:16.376867 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:16.376218 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:16.376867 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.376326 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:16.376867 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.376413 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:16.376867 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.376497 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:16.829487 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:16.829448 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:16.829710 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:16.829497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:16.829710 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.829626 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:16.829710 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.829653 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:16.829710 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.829694 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs podName:86199c5a-74c9-446b-9aa5-028fb5224e6c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:32.829672297 +0000 UTC m=+35.399083058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs") pod "network-metrics-daemon-sr7b5" (UID: "86199c5a-74c9-446b-9aa5-028fb5224e6c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:16.829924 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.829722 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret podName:ba8203bc-44c2-4091-ae5d-f0fb93aba210 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:32.829704241 +0000 UTC m=+35.399115009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret") pod "global-pull-secret-syncer-s29kb" (UID: "ba8203bc-44c2-4091-ae5d-f0fb93aba210") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:16.930798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:16.930755 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:16.930978 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.930900 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:16.930978 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.930925 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:16.930978 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.930940 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5dm5h for pod openshift-network-diagnostics/network-check-target-47glh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:16.931111 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:16.931004 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h podName:3300e5a3-e457-4254-81a3-76412fdb24eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:32.930985535 +0000 UTC m=+35.500396284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dm5h" (UniqueName: "kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h") pod "network-check-target-47glh" (UID: "3300e5a3-e457-4254-81a3-76412fdb24eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:18.376450 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:18.376411 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:18.376924 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:18.376464 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:18.376924 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:18.376412 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:18.376924 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:18.376562 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:18.376924 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:18.376635 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:18.376924 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:18.376730 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:20.375914 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:20.375877 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:20.376455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:20.375875 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:20.376455 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:20.376023 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:20.376455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:20.375885 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:20.376455 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:20.376092 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:20.376455 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:20.376160 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:21.482110 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.481659 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2fd85" event={"ID":"6992d303-6bd8-485d-b532-f782c6809cbc","Type":"ContainerStarted","Data":"876b34a70e6185251096fd66d2d5eddadbb689452f3c76d52b4a311bca751ae4"} Jul 14 17:05:21.483374 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.483345 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wckws" event={"ID":"555eafcf-ccb1-4454-b367-f5b038305022","Type":"ContainerStarted","Data":"e1028b3cd319c7d0aaf33fb1837a932f73fc4f293d1d721c8c665b17abe65c81"} Jul 14 17:05:21.484820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.484797 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pkfmh" event={"ID":"2bc646b1-d627-4d8f-a261-d84275abdb68","Type":"ContainerStarted","Data":"b602eb510d94e38628c8120dbc7a30a76f34b05106de67159eb3882bb407f57f"} Jul 14 17:05:21.486490 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.486462 2558 generic.go:356] "Generic (PLEG): container finished" podID="229304ae-1b47-4f12-b7a5-8e7ce52b0637" containerID="1d9c391bc0f565de9864d85a81a74ea0a5aa181905c71c20e927d0a2fa028e6b" exitCode=0 Jul 14 17:05:21.486607 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.486571 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerDied","Data":"1d9c391bc0f565de9864d85a81a74ea0a5aa181905c71c20e927d0a2fa028e6b"} Jul 14 17:05:21.488913 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.488863 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fscf5" event={"ID":"5e7be499-7243-4991-b4a7-57b1f1070a94","Type":"ContainerStarted","Data":"ac0e5a4a78c0a050318649e1096f1fcb76093258915faabd21b94ca1b9f3369c"} Jul 14 17:05:21.490601 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.490576 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c89hx" event={"ID":"c63b8442-55f0-4748-bd35-9902b296da10","Type":"ContainerStarted","Data":"ae3cf3284a35f84b0d7331c088a8029df63282a340ece00c1236cbeb112fc6d3"} Jul 14 17:05:21.492345 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.492326 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" event={"ID":"750be559-171d-4c27-b164-61129e060388","Type":"ContainerStarted","Data":"282bbcaef63860576fe9d84e77ae02ad16f716ca0dc83106c68f99b36c61dc87"} Jul 14 17:05:21.494915 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.494898 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:05:21.495288 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.495266 2558 generic.go:356] "Generic (PLEG): container finished" podID="3ca33e99-b10d-406f-b4fa-3be9bcbd7467" containerID="52e85cd454634c62bb9759d729091cf58921b7c832b80752aaf0b2771694e8ea" exitCode=1 Jul 14 17:05:21.495373 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.495303 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"32b77f8bbd9afc679061da8bf7a56379634d02d673d4b3cc64b0fda6347b979a"} Jul 14 17:05:21.495429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.495369 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"f514979246af46102493bdfb95140fb1e690f17a01ef35630a1cf30d06c5bd27"} Jul 14 17:05:21.495429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.495420 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerDied","Data":"52e85cd454634c62bb9759d729091cf58921b7c832b80752aaf0b2771694e8ea"} Jul 14 17:05:21.495526 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.495472 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"80649edeb490f0af83ca1d8af6711d7f2963c98826c461dd1176cc979bca570e"} Jul 14 17:05:21.506389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.506337 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2fd85" podStartSLOduration=2.888930297 podStartE2EDuration="22.506319934s" totalImagesPullingTime="19.617389637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:21.505875349 +0000 UTC m=+24.075286120" watchObservedRunningTime="2026-07-14 17:05:21.506319934 +0000 UTC m=+24.075730706" Jul 14 17:05:21.588623 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.588576 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c89hx" podStartSLOduration=2.936289087 podStartE2EDuration="22.58853487s" totalImagesPullingTime="19.652245783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:21.588527122 +0000 UTC m=+24.157937892" watchObservedRunningTime="2026-07-14 17:05:21.58853487 +0000 UTC m=+24.157945640" Jul 14 17:05:21.588784 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.588695 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pkfmh" podStartSLOduration=2.9271289400000002 podStartE2EDuration="22.58868483s" totalImagesPullingTime="19.66155589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:21.563687447 +0000 UTC m=+24.133098215" watchObservedRunningTime="2026-07-14 17:05:21.58868483 +0000 UTC m=+24.158095602" Jul 14 17:05:21.616181 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.616128 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fscf5" podStartSLOduration=2.649331322 podStartE2EDuration="22.616103153s" totalImagesPullingTime="19.966771831s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:21.61564226 +0000 UTC m=+24.185053069" watchObservedRunningTime="2026-07-14 17:05:21.616103153 +0000 UTC m=+24.185513925" Jul 14 17:05:21.640384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:21.640342 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wckws" podStartSLOduration=3.012090556 podStartE2EDuration="22.640307194s" totalImagesPullingTime="19.628216638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:21.640175346 +0000 UTC m=+24.209586117" watchObservedRunningTime="2026-07-14 17:05:21.640307194 +0000 UTC m=+24.209717965" Jul 14 17:05:22.376117 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.376082 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:22.376256 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.376082 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:22.376256 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.376239 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:22.376256 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:22.376241 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:22.376375 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:22.376352 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:22.376446 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:22.376427 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:22.483853 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.483825 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 17:05:22.500075 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.500036 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zq4sn" event={"ID":"7268fe4b-985a-40f4-9eff-317b73bab0cd","Type":"ContainerStarted","Data":"94cda97036571dd83cd19fa0c94afca34076fb4bb4d2aa81bacde3edb0d88ef4"} Jul 14 17:05:22.503418 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.503383 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" event={"ID":"750be559-171d-4c27-b164-61129e060388","Type":"ContainerStarted","Data":"9759ff0085dfa20ac90c1a484739dc2294551935de11f80bf6d2824553cd2391"} Jul 14 17:05:22.507804 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.507783 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:05:22.508270 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.508240 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"02784f9945cfd466e71bfa6bb40456dae3863f493050cd1c97b216280f65c7e4"} Jul 14 17:05:22.508387 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.508323 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"ae692056bc293bae0f9d3a33f1fc53f847ea329d9050669ac9d3585c96bbc108"} Jul 14 17:05:22.527276 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:22.527223 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zq4sn" podStartSLOduration=7.665551737 podStartE2EDuration="22.527201553s" totalImagesPullingTime="14.861649816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:22.526703694 +0000 UTC m=+25.096114490" watchObservedRunningTime="2026-07-14 17:05:22.527201553 +0000 UTC m=+25.096612308" Jul 14 17:05:23.293434 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:23.293319 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T17:05:22.483845341Z","UUID":"7b067e52-a575-42bf-a2df-8664a88127a3","Handler":null,"Name":"","Endpoint":""} Jul 14 17:05:23.295097 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:23.295061 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 17:05:23.295233 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:23.295103 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 17:05:24.375999 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:24.375966 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:24.376464 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:24.376008 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:24.376464 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:24.376072 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:24.376464 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:24.376095 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:24.376464 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:24.376142 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:24.376464 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:24.376250 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:24.516965 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:24.516935 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:05:24.517815 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:24.517674 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"b39adaa708ba3745162d41fe40b2f45310bbbf01c11be27d28af35eb00dcf701"} Jul 14 17:05:25.048493 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:25.048452 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:25.049259 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:25.049235 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:25.520833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:25.520789 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:25.521364 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:25.521346 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wckws" Jul 14 17:05:26.375959 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:26.375736 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:26.376087 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:26.375742 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:26.376087 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:26.375997 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:26.376087 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:26.375768 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:26.376087 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:26.376064 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:26.376234 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:26.376148 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:27.527450 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.527411 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" event={"ID":"750be559-171d-4c27-b164-61129e060388","Type":"ContainerStarted","Data":"a84ae79e306363b0549cacfa06b0ad2ed298cf4be16ae9b1a91436f363559bc8"} Jul 14 17:05:27.530918 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.530756 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:05:27.531260 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.531239 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"a0a59b90581b7335c6e7cb6e48ddce361b312b13849724beb1527847626df841"} Jul 14 17:05:27.531487 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.531464 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:27.531768 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.531658 2558 scope.go:117] "RemoveContainer" containerID="52e85cd454634c62bb9759d729091cf58921b7c832b80752aaf0b2771694e8ea" Jul 14 17:05:27.533168 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.533144 2558 generic.go:356] "Generic (PLEG): container finished" podID="229304ae-1b47-4f12-b7a5-8e7ce52b0637" containerID="33acadc104daa15c8e46d5fc26b142bc62ee0ad3dd3b3a6b41fdf38a8abb1976" exitCode=0 Jul 14 17:05:27.533263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.533188 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerDied","Data":"33acadc104daa15c8e46d5fc26b142bc62ee0ad3dd3b3a6b41fdf38a8abb1976"} Jul 14 17:05:27.549145 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.549125 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:27.549927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:27.549890 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5t6n" podStartSLOduration=4.073765727 podStartE2EDuration="28.549876328s" totalImagesPullingTime="24.476110601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:27.549495264 +0000 UTC m=+30.118906035" watchObservedRunningTime="2026-07-14 17:05:27.549876328 +0000 UTC m=+30.119287098" Jul 14 17:05:28.375852 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.375816 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:28.376136 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.375860 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:28.376136 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.375911 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:28.376136 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:28.375925 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:28.376136 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:28.375988 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:28.376136 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:28.376052 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:28.539936 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.539912 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:05:28.540374 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.540343 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" event={"ID":"3ca33e99-b10d-406f-b4fa-3be9bcbd7467","Type":"ContainerStarted","Data":"1088963d5239dd3712e917d11b102f4d3de966fbb082962d11d2b56870fa2002"} Jul 14 17:05:28.540413 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.540384 2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 17:05:28.540698 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.540679 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:28.543895 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.543872 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s29kb"] Jul 14 17:05:28.543992 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.543956 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:28.544080 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:28.544057 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:28.546372 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.546349 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sr7b5"] Jul 14 17:05:28.546475 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.546447 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:28.546612 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:28.546593 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:28.556801 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.556774 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:28.558027 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.558006 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47glh"] Jul 14 17:05:28.558149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.558093 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:28.558243 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:28.558221 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:28.603149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:28.603099 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" podStartSLOduration=8.946636525 podStartE2EDuration="28.603078782s" totalImagesPullingTime="19.656442257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:28.576438661 +0000 UTC m=+31.145849437" watchObservedRunningTime="2026-07-14 17:05:28.603078782 +0000 UTC m=+31.172489545" Jul 14 17:05:29.544286 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:29.544249 2558 generic.go:356] "Generic (PLEG): container finished" podID="229304ae-1b47-4f12-b7a5-8e7ce52b0637" containerID="0c051ce8a55b9f87f543f8a1646951b22053b220ffb5cd4d7fde9e2bb96814b3" exitCode=0 Jul 14 17:05:29.544742 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:29.544330 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerDied","Data":"0c051ce8a55b9f87f543f8a1646951b22053b220ffb5cd4d7fde9e2bb96814b3"} Jul 14 17:05:29.544742 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:29.544412 2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 17:05:30.376200 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:30.376168 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:30.376404 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:30.376161 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:30.376404 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:30.376284 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:30.376404 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:30.376306 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:30.376588 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:30.376445 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:30.376588 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:30.376523 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:30.547049 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:30.547023 2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 17:05:30.740034 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:30.740000 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:05:31.551611 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:31.551575 2558 generic.go:356] "Generic (PLEG): container finished" podID="229304ae-1b47-4f12-b7a5-8e7ce52b0637" containerID="d545831c1250bbe88b3136e63194057a87c2815b1f02e2805e0107deeb5f7d2e" exitCode=0 Jul 14 17:05:31.552038 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:31.551672 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerDied","Data":"d545831c1250bbe88b3136e63194057a87c2815b1f02e2805e0107deeb5f7d2e"} Jul 14 17:05:31.569262 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:31.569084 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" podUID="3ca33e99-b10d-406f-b4fa-3be9bcbd7467" containerName="ovnkube-controller" probeResult="failure" output="" Jul 14 17:05:32.376117 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:32.376076 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:32.376330 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:32.376076 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:32.376330 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.376196 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s29kb" podUID="ba8203bc-44c2-4091-ae5d-f0fb93aba210" Jul 14 17:05:32.376330 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:32.376210 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:32.376330 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.376286 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-47glh" podUID="3300e5a3-e457-4254-81a3-76412fdb24eb" Jul 14 17:05:32.376475 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.376355 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr7b5" podUID="86199c5a-74c9-446b-9aa5-028fb5224e6c" Jul 14 17:05:32.853240 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:32.853204 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:32.853728 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:32.853269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:32.853728 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.853384 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:32.853728 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.853386 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:32.853728 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.853448 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs podName:86199c5a-74c9-446b-9aa5-028fb5224e6c nodeName:}" failed. No retries permitted until 2026-07-14 17:06:04.853430262 +0000 UTC m=+67.422841029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs") pod "network-metrics-daemon-sr7b5" (UID: "86199c5a-74c9-446b-9aa5-028fb5224e6c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:32.853728 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.853468 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret podName:ba8203bc-44c2-4091-ae5d-f0fb93aba210 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:04.853457396 +0000 UTC m=+67.422868360 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret") pod "global-pull-secret-syncer-s29kb" (UID: "ba8203bc-44c2-4091-ae5d-f0fb93aba210") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:32.954464 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:32.954423 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:32.954651 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.954578 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:32.954651 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.954597 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:32.954651 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.954606 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5dm5h for pod openshift-network-diagnostics/network-check-target-47glh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:32.954769 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:32.954662 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h podName:3300e5a3-e457-4254-81a3-76412fdb24eb nodeName:}" failed. No retries permitted until 2026-07-14 17:06:04.954646708 +0000 UTC m=+67.524057463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5dm5h" (UniqueName: "kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h") pod "network-check-target-47glh" (UID: "3300e5a3-e457-4254-81a3-76412fdb24eb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:34.280968 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.280936 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-240.ec2.internal" event="NodeReady" Jul 14 17:05:34.281395 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.281084 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 17:05:34.321930 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.321889 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fb86598f-m8z72"] Jul 14 17:05:34.325461 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.325279 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.328110 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.328085 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 17:05:34.328383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.328355 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 17:05:34.328628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.328608 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 17:05:34.328744 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.328670 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-888d9\"" Jul 14 17:05:34.335079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.335058 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 17:05:34.340318 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.340247 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb86598f-m8z72"] Jul 14 17:05:34.340861 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.340445 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj"] Jul 14 17:05:34.342602 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.342578 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7g9zk"] Jul 14 17:05:34.342910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.342892 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" Jul 14 17:05:34.344450 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.344430 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm"] Jul 14 17:05:34.344559 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.344526 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.345748 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.345725 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.345842 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.345739 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g7rcw\"" Jul 14 17:05:34.345842 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.345727 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.346780 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.346763 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-lpj7g"] Jul 14 17:05:34.346917 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.346861 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:34.347193 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.347160 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8v8vb\"" Jul 14 17:05:34.347307 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.347288 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 17:05:34.347406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.347395 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 17:05:34.349367 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.349346 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl"] Jul 14 17:05:34.349521 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.349501 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.350198 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.350137 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.350198 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.350171 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.350198 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.350179 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 14 17:05:34.350391 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.350208 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fssx8\"" Jul 14 17:05:34.351384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.351368 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.353146 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.353129 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-t8nwk\"" Jul 14 17:05:34.353658 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.353135 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 14 17:05:34.353658 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.353140 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj"] Jul 14 17:05:34.354326 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.354309 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.354953 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.354926 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 17:05:34.355182 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.354619 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 14 17:05:34.355315 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.354568 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.355315 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.354761 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-m9bnz\"" Jul 14 17:05:34.355443 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.354629 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 17:05:34.355716 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.355683 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.355920 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.355904 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.358830 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.358810 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl"] Jul 14 17:05:34.359841 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.359824 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bf2qg"] Jul 14 17:05:34.362439 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.362419 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm"] Jul 14 17:05:34.362566 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.362471 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k"] Jul 14 17:05:34.362566 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.362512 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 14 17:05:34.362682 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.362586 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:34.364581 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.364536 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7g9zk"] Jul 14 17:05:34.364680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.364608 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zlgcw"] Jul 14 17:05:34.364680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.364627 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.367274 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.367248 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 14 17:05:34.367384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.367294 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 14 17:05:34.367384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.367347 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.369059 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.368273 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cbcjd\"" Jul 14 17:05:34.369265 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.369189 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.374433 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.374413 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 14 17:05:34.374931 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.374914 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-c9v8j\"" Jul 14 17:05:34.375302 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.375285 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.376027 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7497bbf74b-2lkjk"] Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.377219 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.377926 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2"] Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.378147 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.378603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.379023 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.379220 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:05:34.381151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.380907 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl"] Jul 14 17:05:34.381560 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.381285 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.381560 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.381530 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:05:34.382072 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.382046 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 17:05:34.383967 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.382075 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.384085 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.382666 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.384148 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.382790 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 14 17:05:34.384219 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383013 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.384311 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383160 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c4hcb"] Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.384393 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383255 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383282 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383343 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383393 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zqz5t\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383435 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bnmjj\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383477 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383483 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383510 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 14 17:05:34.385025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383578 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.385524 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383582 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tv9jb\"" Jul 14 17:05:34.385524 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.385191 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4z8fb\"" Jul 14 17:05:34.385524 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383741 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7wgm\"" Jul 14 17:05:34.385524 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383671 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 14 17:05:34.385751 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.383235 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" Jul 14 17:05:34.385874 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.385856 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.385955 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.385886 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 14 17:05:34.386190 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.386174 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 14 17:05:34.386716 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.386697 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k"] Jul 14 17:05:34.386977 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.386878 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:34.387212 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.387195 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bf2qg"] Jul 14 17:05:34.387359 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.387341 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zlgcw"] Jul 14 17:05:34.387455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.387389 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2"] Jul 14 17:05:34.387455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.387429 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl"] Jul 14 17:05:34.387455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.387449 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-lpj7g"] Jul 14 17:05:34.389465 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.389444 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 17:05:34.390144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.390125 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-t6h6d\"" Jul 14 17:05:34.390553 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.390514 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 17:05:34.390650 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.390583 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 17:05:34.390650 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.390584 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 17:05:34.390852 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.390832 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vhkzd\"" Jul 14 17:05:34.403808 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.403758 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c4hcb"] Jul 14 17:05:34.404608 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.404590 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7497bbf74b-2lkjk"] Jul 14 17:05:34.464671 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464640 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-config\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.464808 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464683 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2be7952-ab97-43f0-a8b5-1f227c22b10c-trusted-ca\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.464808 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5jmf\" (UniqueName: \"kubernetes.io/projected/523c6d19-8d88-45d5-9560-8f32e260aa1b-kube-api-access-f5jmf\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:34.464808 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464733 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-config-volume\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.464808 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464793 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.465030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464825 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kcqj\" (UniqueName: \"kubernetes.io/projected/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-kube-api-access-6kcqj\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.465030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464851 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c2be7952-ab97-43f0-a8b5-1f227c22b10c-config\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.465030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464902 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e812b93e-638c-429e-93cd-c4fa57fc3f73-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:34.465030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.464976 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-certificates\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.465220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465030 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df83cbc3-0758-4506-8e21-6641839d5f8b-snapshots\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.465220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465056 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-stats-auth\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.465220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465080 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-installation-pull-secrets\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.465220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465138 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-default-certificate\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.465220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465209 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf4ww\" (UniqueName: \"kubernetes.io/projected/fef52ed7-0972-4f2e-8cc0-2780b7f59855-kube-api-access-hf4ww\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.465877 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75d34625-d739-4296-be29-f90c173c1a9c-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.465987 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465912 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.465987 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465945 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-bound-sa-token\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.465987 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465972 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2t6c\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-kube-api-access-k2t6c\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.466135 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.465997 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc527\" (UniqueName: \"kubernetes.io/projected/c2be7952-ab97-43f0-a8b5-1f227c22b10c-kube-api-access-fc527\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.466135 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466034 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75d34625-d739-4296-be29-f90c173c1a9c-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.466135 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466068 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:34.466135 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466089 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc87436a-95c3-40e5-9470-6acaf43aabbe-ca-trust-extracted\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.466135 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tllgx\" (UniqueName: \"kubernetes.io/projected/75d34625-d739-4296-be29-f90c173c1a9c-kube-api-access-tllgx\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.466384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466166 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tbvn\" (UniqueName: \"kubernetes.io/projected/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-kube-api-access-5tbvn\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.466384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466217 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df83cbc3-0758-4506-8e21-6641839d5f8b-tmp\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.466384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2be7952-ab97-43f0-a8b5-1f227c22b10c-serving-cert\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.466384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466312 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df83cbc3-0758-4506-8e21-6641839d5f8b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.466592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466393 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:34.466592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466419 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.466592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466460 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g85mj\" (UniqueName: \"kubernetes.io/projected/e4845b9f-e512-48e6-b8d4-2ae25652b0d0-kube-api-access-g85mj\") pod \"volume-data-source-validator-78495597f6-g2gjj\" (UID: \"e4845b9f-e512-48e6-b8d4-2ae25652b0d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" Jul 14 17:05:34.466592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466484 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-tmp-dir\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.466592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466524 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-serving-cert\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.466592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466580 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpfr6\" (UniqueName: \"kubernetes.io/projected/df83cbc3-0758-4506-8e21-6641839d5f8b-kube-api-access-jpfr6\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.466798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466638 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-image-registry-private-configuration\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.466798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466666 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-trusted-ca\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.466798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466690 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.466798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466714 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df83cbc3-0758-4506-8e21-6641839d5f8b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.466798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.466737 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df83cbc3-0758-4506-8e21-6641839d5f8b-serving-cert\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.567364 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567273 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc527\" (UniqueName: \"kubernetes.io/projected/c2be7952-ab97-43f0-a8b5-1f227c22b10c-kube-api-access-fc527\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.567364 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567318 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75d34625-d739-4296-be29-f90c173c1a9c-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.567364 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567355 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:34.567684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc87436a-95c3-40e5-9470-6acaf43aabbe-ca-trust-extracted\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.567684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567404 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tllgx\" (UniqueName: \"kubernetes.io/projected/75d34625-d739-4296-be29-f90c173c1a9c-kube-api-access-tllgx\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.567684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567441 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtkmq\" (UniqueName: \"kubernetes.io/projected/5c3c9438-260b-4e36-91c9-d6d8b300965d-kube-api-access-gtkmq\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.567684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567500 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbfmh\" (UniqueName: \"kubernetes.io/projected/2f955fd3-d01a-4775-8f30-a6d37287c0e0-kube-api-access-zbfmh\") pod \"ingress-canary-c4hcb\" (UID: \"2f955fd3-d01a-4775-8f30-a6d37287c0e0\") " pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:34.567684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567553 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tbvn\" (UniqueName: \"kubernetes.io/projected/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-kube-api-access-5tbvn\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.567684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567573 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df83cbc3-0758-4506-8e21-6641839d5f8b-tmp\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.567684 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.567573 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567797 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2be7952-ab97-43f0-a8b5-1f227c22b10c-serving-cert\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567839 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df83cbc3-0758-4506-8e21-6641839d5f8b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567887 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567913 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567946 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g85mj\" (UniqueName: \"kubernetes.io/projected/e4845b9f-e512-48e6-b8d4-2ae25652b0d0-kube-api-access-g85mj\") pod \"volume-data-source-validator-78495597f6-g2gjj\" (UID: \"e4845b9f-e512-48e6-b8d4-2ae25652b0d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567969 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-tmp-dir\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.567992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-serving-cert\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.568016 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568018 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpfr6\" (UniqueName: \"kubernetes.io/projected/df83cbc3-0758-4506-8e21-6641839d5f8b-kube-api-access-jpfr6\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.568266 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568042 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c3c9438-260b-4e36-91c9-d6d8b300965d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.568266 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568068 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df83cbc3-0758-4506-8e21-6641839d5f8b-tmp\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.568266 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.568155 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls podName:523c6d19-8d88-45d5-9560-8f32e260aa1b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:35.068130044 +0000 UTC m=+37.637540807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls") pod "cluster-samples-operator-65db858894-z84mm" (UID: "523c6d19-8d88-45d5-9560-8f32e260aa1b") : secret "samples-operator-tls" not found Jul 14 17:05:34.568266 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.568153 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:34.568266 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.568199 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls podName:5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f nodeName:}" failed. No retries permitted until 2026-07-14 17:05:35.068186551 +0000 UTC m=+37.637597303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls") pod "dns-default-7g9zk" (UID: "5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f") : secret "dns-default-metrics-tls" not found Jul 14 17:05:34.568266 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.568208 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 17:05:34.568266 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.568268 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert podName:e812b93e-638c-429e-93cd-c4fa57fc3f73 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:35.068249806 +0000 UTC m=+37.637660572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bf2qg" (UID: "e812b93e-638c-429e-93cd-c4fa57fc3f73") : secret "networking-console-plugin-cert" not found Jul 14 17:05:34.568489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568410 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc87436a-95c3-40e5-9470-6acaf43aabbe-ca-trust-extracted\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.568489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568406 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75d34625-d739-4296-be29-f90c173c1a9c-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.568489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568073 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-image-registry-private-configuration\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.568489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568484 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-trusted-ca\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568513 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568555 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-tmp-dir\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568562 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df83cbc3-0758-4506-8e21-6641839d5f8b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568589 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df83cbc3-0758-4506-8e21-6641839d5f8b-serving-cert\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568624 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-config\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2be7952-ab97-43f0-a8b5-1f227c22b10c-trusted-ca\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.568693 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:35.06865416 +0000 UTC m=+37.638064928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : configmap references non-existent config key: service-ca.crt Jul 14 17:05:34.568777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568726 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmz8v\" (UniqueName: \"kubernetes.io/projected/22e1d58b-ca34-4279-9ece-d1062a82451c-kube-api-access-qmz8v\") pod \"network-check-source-869dcfcdcd-2mrxl\" (UID: \"22e1d58b-ca34-4279-9ece-d1062a82451c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568790 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f955fd3-d01a-4775-8f30-a6d37287c0e0-cert\") pod \"ingress-canary-c4hcb\" (UID: \"2f955fd3-d01a-4775-8f30-a6d37287c0e0\") " pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568854 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5jmf\" (UniqueName: \"kubernetes.io/projected/523c6d19-8d88-45d5-9560-8f32e260aa1b-kube-api-access-f5jmf\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568879 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-config-volume\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568924 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.568978 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569017 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kcqj\" (UniqueName: \"kubernetes.io/projected/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-kube-api-access-6kcqj\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569040 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c2be7952-ab97-43f0-a8b5-1f227c22b10c-config\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e812b93e-638c-429e-93cd-c4fa57fc3f73-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569086 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-certificates\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569120 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df83cbc3-0758-4506-8e21-6641839d5f8b-snapshots\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569143 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-stats-auth\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-installation-pull-secrets\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.569624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569207 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-default-certificate\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569233 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf4ww\" (UniqueName: \"kubernetes.io/projected/fef52ed7-0972-4f2e-8cc0-2780b7f59855-kube-api-access-hf4ww\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569257 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75d34625-d739-4296-be29-f90c173c1a9c-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569274 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df83cbc3-0758-4506-8e21-6641839d5f8b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-bound-sa-token\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569363 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2t6c\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-kube-api-access-k2t6c\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569630 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-trusted-ca\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.569630 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c2be7952-ab97-43f0-a8b5-1f227c22b10c-trusted-ca\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.570104 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-certificates\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.570268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.570165 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df83cbc3-0758-4506-8e21-6641839d5f8b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.570772 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.570275 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-config-volume\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.570772 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.570379 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 17:05:34.570772 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.570450 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:35.070432956 +0000 UTC m=+37.639843709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : secret "router-metrics-certs-default" not found Jul 14 17:05:34.572132 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.571240 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c2be7952-ab97-43f0-a8b5-1f227c22b10c-config\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.572132 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.571360 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:34.572132 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.571375 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb86598f-m8z72: secret "image-registry-tls" not found Jul 14 17:05:34.572132 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.571417 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls podName:bc87436a-95c3-40e5-9470-6acaf43aabbe nodeName:}" failed. No retries permitted until 2026-07-14 17:05:35.071401996 +0000 UTC m=+37.640812749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls") pod "image-registry-6fb86598f-m8z72" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe") : secret "image-registry-tls" not found Jul 14 17:05:34.572132 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.571635 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-config\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.572132 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.571809 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e812b93e-638c-429e-93cd-c4fa57fc3f73-nginx-conf\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:34.572132 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.572088 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df83cbc3-0758-4506-8e21-6641839d5f8b-snapshots\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.573845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.573822 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/75d34625-d739-4296-be29-f90c173c1a9c-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.574456 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.574218 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-installation-pull-secrets\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.574620 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.574595 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c2be7952-ab97-43f0-a8b5-1f227c22b10c-serving-cert\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.574743 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.574634 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-image-registry-private-configuration\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.574743 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.574725 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-serving-cert\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.575153 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.575114 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-stats-auth\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.575494 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.575476 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-default-certificate\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.576181 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.576130 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df83cbc3-0758-4506-8e21-6641839d5f8b-serving-cert\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.579807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.579780 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc527\" (UniqueName: \"kubernetes.io/projected/c2be7952-ab97-43f0-a8b5-1f227c22b10c-kube-api-access-fc527\") pod \"console-operator-7866497494-lpj7g\" (UID: \"c2be7952-ab97-43f0-a8b5-1f227c22b10c\") " pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.580203 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.580181 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tbvn\" (UniqueName: \"kubernetes.io/projected/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-kube-api-access-5tbvn\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:34.581652 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.581632 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g85mj\" (UniqueName: \"kubernetes.io/projected/e4845b9f-e512-48e6-b8d4-2ae25652b0d0-kube-api-access-g85mj\") pod \"volume-data-source-validator-78495597f6-g2gjj\" (UID: \"e4845b9f-e512-48e6-b8d4-2ae25652b0d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" Jul 14 17:05:34.582450 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.582217 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tllgx\" (UniqueName: \"kubernetes.io/projected/75d34625-d739-4296-be29-f90c173c1a9c-kube-api-access-tllgx\") pod \"kube-storage-version-migrator-operator-799cb7f958-px8zl\" (UID: \"75d34625-d739-4296-be29-f90c173c1a9c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.582450 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.582234 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf4ww\" (UniqueName: \"kubernetes.io/projected/fef52ed7-0972-4f2e-8cc0-2780b7f59855-kube-api-access-hf4ww\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:34.582740 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.582709 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpfr6\" (UniqueName: \"kubernetes.io/projected/df83cbc3-0758-4506-8e21-6641839d5f8b-kube-api-access-jpfr6\") pod \"insights-operator-68d9dd94cf-zlgcw\" (UID: \"df83cbc3-0758-4506-8e21-6641839d5f8b\") " pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.584633 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.584534 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5jmf\" (UniqueName: \"kubernetes.io/projected/523c6d19-8d88-45d5-9560-8f32e260aa1b-kube-api-access-f5jmf\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:34.584870 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.584826 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2t6c\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-kube-api-access-k2t6c\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.584960 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.584906 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-bound-sa-token\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:34.585831 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.585807 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kcqj\" (UniqueName: \"kubernetes.io/projected/f42ac6be-eba5-4f04-99d6-be1b2e3c9e98-kube-api-access-6kcqj\") pod \"service-ca-operator-7695fc6c8c-wgq9k\" (UID: \"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.656342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.656302 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" Jul 14 17:05:34.670182 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.670148 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c3c9438-260b-4e36-91c9-d6d8b300965d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.670319 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.670207 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmz8v\" (UniqueName: \"kubernetes.io/projected/22e1d58b-ca34-4279-9ece-d1062a82451c-kube-api-access-qmz8v\") pod \"network-check-source-869dcfcdcd-2mrxl\" (UID: \"22e1d58b-ca34-4279-9ece-d1062a82451c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" Jul 14 17:05:34.670319 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.670235 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f955fd3-d01a-4775-8f30-a6d37287c0e0-cert\") pod \"ingress-canary-c4hcb\" (UID: \"2f955fd3-d01a-4775-8f30-a6d37287c0e0\") " pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:34.670319 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.670271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.670472 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.670395 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:34.670472 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:34.670459 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls podName:5c3c9438-260b-4e36-91c9-d6d8b300965d nodeName:}" failed. No retries permitted until 2026-07-14 17:05:35.170438627 +0000 UTC m=+37.739849376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-b4wp2" (UID: "5c3c9438-260b-4e36-91c9-d6d8b300965d") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:34.670688 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.670663 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtkmq\" (UniqueName: \"kubernetes.io/projected/5c3c9438-260b-4e36-91c9-d6d8b300965d-kube-api-access-gtkmq\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.670779 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.670706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbfmh\" (UniqueName: \"kubernetes.io/projected/2f955fd3-d01a-4775-8f30-a6d37287c0e0-kube-api-access-zbfmh\") pod \"ingress-canary-c4hcb\" (UID: \"2f955fd3-d01a-4775-8f30-a6d37287c0e0\") " pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:34.671013 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.670978 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c3c9438-260b-4e36-91c9-d6d8b300965d-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.672970 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.672951 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f955fd3-d01a-4775-8f30-a6d37287c0e0-cert\") pod \"ingress-canary-c4hcb\" (UID: \"2f955fd3-d01a-4775-8f30-a6d37287c0e0\") " pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:34.682202 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.682148 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmz8v\" (UniqueName: \"kubernetes.io/projected/22e1d58b-ca34-4279-9ece-d1062a82451c-kube-api-access-qmz8v\") pod \"network-check-source-869dcfcdcd-2mrxl\" (UID: \"22e1d58b-ca34-4279-9ece-d1062a82451c\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" Jul 14 17:05:34.682202 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.682182 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbfmh\" (UniqueName: \"kubernetes.io/projected/2f955fd3-d01a-4775-8f30-a6d37287c0e0-kube-api-access-zbfmh\") pod \"ingress-canary-c4hcb\" (UID: \"2f955fd3-d01a-4775-8f30-a6d37287c0e0\") " pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:34.682828 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.682804 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtkmq\" (UniqueName: \"kubernetes.io/projected/5c3c9438-260b-4e36-91c9-d6d8b300965d-kube-api-access-gtkmq\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:34.691684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.691659 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:34.697720 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.697694 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" Jul 14 17:05:34.714270 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.714225 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" Jul 14 17:05:34.720685 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.720659 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" Jul 14 17:05:34.769252 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.769216 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" Jul 14 17:05:34.775236 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:34.775204 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c4hcb" Jul 14 17:05:35.074616 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.074565 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:35.074815 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.074653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:35.074815 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.074708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:35.074815 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074753 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 17:05:35.074815 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.074757 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:35.074815 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074811 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074823 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074831 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:36.074809133 +0000 UTC m=+38.644219896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : secret "router-metrics-certs-default" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074760 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074869 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074882 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb86598f-m8z72: secret "image-registry-tls" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.074811 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074867 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls podName:523c6d19-8d88-45d5-9560-8f32e260aa1b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:36.074852332 +0000 UTC m=+38.644263082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls") pod "cluster-samples-operator-65db858894-z84mm" (UID: "523c6d19-8d88-45d5-9560-8f32e260aa1b") : secret "samples-operator-tls" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074952 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert podName:e812b93e-638c-429e-93cd-c4fa57fc3f73 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:36.074935915 +0000 UTC m=+38.644346669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bf2qg" (UID: "e812b93e-638c-429e-93cd-c4fa57fc3f73") : secret "networking-console-plugin-cert" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074973 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls podName:5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f nodeName:}" failed. No retries permitted until 2026-07-14 17:05:36.074963353 +0000 UTC m=+38.644374104 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls") pod "dns-default-7g9zk" (UID: "5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f") : secret "dns-default-metrics-tls" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.074993 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls podName:bc87436a-95c3-40e5-9470-6acaf43aabbe nodeName:}" failed. No retries permitted until 2026-07-14 17:05:36.074984173 +0000 UTC m=+38.644394928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls") pod "image-registry-6fb86598f-m8z72" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe") : secret "image-registry-tls" not found Jul 14 17:05:35.075065 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.075051 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:35.075487 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.075155 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:36.075139826 +0000 UTC m=+38.644550610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : configmap references non-existent config key: service-ca.crt Jul 14 17:05:35.176392 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.176347 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:35.176594 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.176524 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:35.176656 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:35.176628 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls podName:5c3c9438-260b-4e36-91c9-d6d8b300965d nodeName:}" failed. No retries permitted until 2026-07-14 17:05:36.176601333 +0000 UTC m=+38.746012083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-b4wp2" (UID: "5c3c9438-260b-4e36-91c9-d6d8b300965d") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:35.884366 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.884085 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl"] Jul 14 17:05:35.888360 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.887287 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zlgcw"] Jul 14 17:05:35.895337 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:35.892719 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d34625_d739_4296_be29_f90c173c1a9c.slice/crio-2b26d80bdb57667fbae5ecd11aa5ef90015385805c945662b714a69c64fa450d WatchSource:0}: Error finding container 2b26d80bdb57667fbae5ecd11aa5ef90015385805c945662b714a69c64fa450d: Status 404 returned error can't find the container with id 2b26d80bdb57667fbae5ecd11aa5ef90015385805c945662b714a69c64fa450d Jul 14 17:05:35.895337 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.895200 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k"] Jul 14 17:05:35.905643 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.905614 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c4hcb"] Jul 14 17:05:35.907825 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.907799 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-lpj7g"] Jul 14 17:05:35.912927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.912903 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj"] Jul 14 17:05:35.919365 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:35.919339 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl"] Jul 14 17:05:35.924766 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:35.922996 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2be7952_ab97_43f0_a8b5_1f227c22b10c.slice/crio-d30fdfbe41f25aafbec0051a44765684bafcd3dffbef4d5e73747d751099a77c WatchSource:0}: Error finding container d30fdfbe41f25aafbec0051a44765684bafcd3dffbef4d5e73747d751099a77c: Status 404 returned error can't find the container with id d30fdfbe41f25aafbec0051a44765684bafcd3dffbef4d5e73747d751099a77c Jul 14 17:05:35.925484 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:35.925429 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4845b9f_e512_48e6_b8d4_2ae25652b0d0.slice/crio-ef6b7742463a712567ccdb23eb053302513444f2e314d4b78647d43c45baaca1 WatchSource:0}: Error finding container ef6b7742463a712567ccdb23eb053302513444f2e314d4b78647d43c45baaca1: Status 404 returned error can't find the container with id ef6b7742463a712567ccdb23eb053302513444f2e314d4b78647d43c45baaca1 Jul 14 17:05:35.926733 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:35.926643 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e1d58b_ca34_4279_9ece_d1062a82451c.slice/crio-67648227daafad3258abe61b6be096f03d5d8601ce787d01eab63e91a6cfd686 WatchSource:0}: Error finding container 67648227daafad3258abe61b6be096f03d5d8601ce787d01eab63e91a6cfd686: Status 404 returned error can't find the container with id 67648227daafad3258abe61b6be096f03d5d8601ce787d01eab63e91a6cfd686 Jul 14 17:05:35.927747 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:35.927466 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f955fd3_d01a_4775_8f30_a6d37287c0e0.slice/crio-8a2396cb1c52c4f2f8b2fa2d34ea48d37081e95a1861352d7886f006dce09c0f WatchSource:0}: Error finding container 8a2396cb1c52c4f2f8b2fa2d34ea48d37081e95a1861352d7886f006dce09c0f: Status 404 returned error can't find the container with id 8a2396cb1c52c4f2f8b2fa2d34ea48d37081e95a1861352d7886f006dce09c0f Jul 14 17:05:35.928817 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:35.928307 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf83cbc3_0758_4506_8e21_6641839d5f8b.slice/crio-2ec27e85871730febdf7f301742598b45ccd825578e0ceac4e2e2aafd2180bc4 WatchSource:0}: Error finding container 2ec27e85871730febdf7f301742598b45ccd825578e0ceac4e2e2aafd2180bc4: Status 404 returned error can't find the container with id 2ec27e85871730febdf7f301742598b45ccd825578e0ceac4e2e2aafd2180bc4 Jul 14 17:05:35.929656 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:05:35.929406 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42ac6be_eba5_4f04_99d6_be1b2e3c9e98.slice/crio-00c2566a3dab0003d57c38d1f8a2888236723b58b413a3a4a8b49ba4083ae8ee WatchSource:0}: Error finding container 00c2566a3dab0003d57c38d1f8a2888236723b58b413a3a4a8b49ba4083ae8ee: Status 404 returned error can't find the container with id 00c2566a3dab0003d57c38d1f8a2888236723b58b413a3a4a8b49ba4083ae8ee Jul 14 17:05:36.086559 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.086512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:36.086748 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.086587 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:36.086748 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.086676 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 17:05:36.086748 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.086698 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:36.086748 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.086748 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert podName:e812b93e-638c-429e-93cd-c4fa57fc3f73 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:38.086727983 +0000 UTC m=+40.656138741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bf2qg" (UID: "e812b93e-638c-429e-93cd-c4fa57fc3f73") : secret "networking-console-plugin-cert" not found Jul 14 17:05:36.086883 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.086676 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:36.086883 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.086786 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:38.086775966 +0000 UTC m=+40.656186729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : configmap references non-existent config key: service-ca.crt Jul 14 17:05:36.086950 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.086890 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls podName:5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f nodeName:}" failed. No retries permitted until 2026-07-14 17:05:38.086871117 +0000 UTC m=+40.656281869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls") pod "dns-default-7g9zk" (UID: "5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f") : secret "dns-default-metrics-tls" not found Jul 14 17:05:36.086950 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.086932 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:36.087008 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.086989 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:36.087039 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.087023 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:36.087080 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.087066 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 17:05:36.087122 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.087106 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:38.087096648 +0000 UTC m=+40.656507400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : secret "router-metrics-certs-default" not found Jul 14 17:05:36.087163 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.087126 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:05:36.087163 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.087143 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:36.087163 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.087156 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls podName:523c6d19-8d88-45d5-9560-8f32e260aa1b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:38.087145334 +0000 UTC m=+40.656556084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls") pod "cluster-samples-operator-65db858894-z84mm" (UID: "523c6d19-8d88-45d5-9560-8f32e260aa1b") : secret "samples-operator-tls" not found Jul 14 17:05:36.087163 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.087159 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb86598f-m8z72: secret "image-registry-tls" not found Jul 14 17:05:36.087337 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.087196 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls podName:bc87436a-95c3-40e5-9470-6acaf43aabbe nodeName:}" failed. No retries permitted until 2026-07-14 17:05:38.087189262 +0000 UTC m=+40.656600011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls") pod "image-registry-6fb86598f-m8z72" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe") : secret "image-registry-tls" not found Jul 14 17:05:36.187861 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.187759 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:36.188022 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.187929 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:36.188022 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:36.188009 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls podName:5c3c9438-260b-4e36-91c9-d6d8b300965d nodeName:}" failed. No retries permitted until 2026-07-14 17:05:38.187988582 +0000 UTC m=+40.757399336 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-b4wp2" (UID: "5c3c9438-260b-4e36-91c9-d6d8b300965d") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:36.572241 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.572202 2558 generic.go:356] "Generic (PLEG): container finished" podID="229304ae-1b47-4f12-b7a5-8e7ce52b0637" containerID="dfaf3637c256a5189afacb0e2a7e9556c8a580f5cd84041f5cbc554325f03f34" exitCode=0 Jul 14 17:05:36.572432 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.572331 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerDied","Data":"dfaf3637c256a5189afacb0e2a7e9556c8a580f5cd84041f5cbc554325f03f34"} Jul 14 17:05:36.578243 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.577798 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" event={"ID":"df83cbc3-0758-4506-8e21-6641839d5f8b","Type":"ContainerStarted","Data":"2ec27e85871730febdf7f301742598b45ccd825578e0ceac4e2e2aafd2180bc4"} Jul 14 17:05:36.581336 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.581269 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" event={"ID":"75d34625-d739-4296-be29-f90c173c1a9c","Type":"ContainerStarted","Data":"2b26d80bdb57667fbae5ecd11aa5ef90015385805c945662b714a69c64fa450d"} Jul 14 17:05:36.584633 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.584587 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c4hcb" event={"ID":"2f955fd3-d01a-4775-8f30-a6d37287c0e0","Type":"ContainerStarted","Data":"8a2396cb1c52c4f2f8b2fa2d34ea48d37081e95a1861352d7886f006dce09c0f"} Jul 14 17:05:36.586141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.586079 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" event={"ID":"e4845b9f-e512-48e6-b8d4-2ae25652b0d0","Type":"ContainerStarted","Data":"ef6b7742463a712567ccdb23eb053302513444f2e314d4b78647d43c45baaca1"} Jul 14 17:05:36.588994 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.588934 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" event={"ID":"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98","Type":"ContainerStarted","Data":"00c2566a3dab0003d57c38d1f8a2888236723b58b413a3a4a8b49ba4083ae8ee"} Jul 14 17:05:36.591409 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.591349 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lpj7g" event={"ID":"c2be7952-ab97-43f0-a8b5-1f227c22b10c","Type":"ContainerStarted","Data":"d30fdfbe41f25aafbec0051a44765684bafcd3dffbef4d5e73747d751099a77c"} Jul 14 17:05:36.593087 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:36.593054 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" event={"ID":"22e1d58b-ca34-4279-9ece-d1062a82451c","Type":"ContainerStarted","Data":"67648227daafad3258abe61b6be096f03d5d8601ce787d01eab63e91a6cfd686"} Jul 14 17:05:37.601040 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:37.601001 2558 generic.go:356] "Generic (PLEG): container finished" podID="229304ae-1b47-4f12-b7a5-8e7ce52b0637" containerID="dceda606a63664824fc17bf5196cae3c5cbab5fc7c8f733b834fc658f8954ced" exitCode=0 Jul 14 17:05:37.601617 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:37.601059 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerDied","Data":"dceda606a63664824fc17bf5196cae3c5cbab5fc7c8f733b834fc658f8954ced"} Jul 14 17:05:38.111683 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.111520 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:38.111683 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.111606 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:38.111683 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.111652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111693 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.111717 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111754 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111772 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert podName:e812b93e-638c-429e-93cd-c4fa57fc3f73 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:42.11175013 +0000 UTC m=+44.681160884 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bf2qg" (UID: "e812b93e-638c-429e-93cd-c4fa57fc3f73") : secret "networking-console-plugin-cert" not found Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111816 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls podName:5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f nodeName:}" failed. No retries permitted until 2026-07-14 17:05:42.111805503 +0000 UTC m=+44.681216252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls") pod "dns-default-7g9zk" (UID: "5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f") : secret "dns-default-metrics-tls" not found Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111816 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111864 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:42.111847167 +0000 UTC m=+44.681257931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : secret "router-metrics-certs-default" not found Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.111885 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111935 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.111937 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:38.111958 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111949 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb86598f-m8z72: secret "image-registry-tls" not found Jul 14 17:05:38.112484 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.111986 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls podName:bc87436a-95c3-40e5-9470-6acaf43aabbe nodeName:}" failed. No retries permitted until 2026-07-14 17:05:42.111972066 +0000 UTC m=+44.681382840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls") pod "image-registry-6fb86598f-m8z72" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe") : secret "image-registry-tls" not found Jul 14 17:05:38.112484 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.112008 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:42.111999084 +0000 UTC m=+44.681409833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : configmap references non-existent config key: service-ca.crt Jul 14 17:05:38.112484 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.112027 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:05:38.112484 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.112062 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls podName:523c6d19-8d88-45d5-9560-8f32e260aa1b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:42.112051493 +0000 UTC m=+44.681462256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls") pod "cluster-samples-operator-65db858894-z84mm" (UID: "523c6d19-8d88-45d5-9560-8f32e260aa1b") : secret "samples-operator-tls" not found Jul 14 17:05:38.213250 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.212568 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:38.213250 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.212771 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:38.213250 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:38.212838 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls podName:5c3c9438-260b-4e36-91c9-d6d8b300965d nodeName:}" failed. No retries permitted until 2026-07-14 17:05:42.21281884 +0000 UTC m=+44.782229604 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-b4wp2" (UID: "5c3c9438-260b-4e36-91c9-d6d8b300965d") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:38.613050 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.612967 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" event={"ID":"229304ae-1b47-4f12-b7a5-8e7ce52b0637","Type":"ContainerStarted","Data":"421cec5e0d617a3572c35e5b59c35bea599310bd0af7d1162fb5841513d48ac9"} Jul 14 17:05:38.641606 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:38.641536 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vr2g8" podStartSLOduration=8.747167421 podStartE2EDuration="39.641515448s" totalImagesPullingTime="30.894348027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:38.64014225 +0000 UTC m=+41.209553044" watchObservedRunningTime="2026-07-14 17:05:38.641515448 +0000 UTC m=+41.210926220" Jul 14 17:05:42.151674 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:42.151622 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:42.151711 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:42.151738 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:42.151781 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.151812 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:42.151841 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.151841 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.151887 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.151891 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls podName:523c6d19-8d88-45d5-9560-8f32e260aa1b nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.151867734 +0000 UTC m=+52.721278507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls") pod "cluster-samples-operator-65db858894-z84mm" (UID: "523c6d19-8d88-45d5-9560-8f32e260aa1b") : secret "samples-operator-tls" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.151964 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.152000 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.151985413 +0000 UTC m=+52.721396192 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : secret "router-metrics-certs-default" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:42.151963 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.152020 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.152035 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb86598f-m8z72: secret "image-registry-tls" not found Jul 14 17:05:42.152372 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.152039 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert podName:e812b93e-638c-429e-93cd-c4fa57fc3f73 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.152006181 +0000 UTC m=+52.721416943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bf2qg" (UID: "e812b93e-638c-429e-93cd-c4fa57fc3f73") : secret "networking-console-plugin-cert" not found Jul 14 17:05:42.152977 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.152080 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.152068744 +0000 UTC m=+52.721479500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : configmap references non-existent config key: service-ca.crt Jul 14 17:05:42.152977 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.152107 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls podName:5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.152096261 +0000 UTC m=+52.721507025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls") pod "dns-default-7g9zk" (UID: "5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f") : secret "dns-default-metrics-tls" not found Jul 14 17:05:42.152977 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.152160 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls podName:bc87436a-95c3-40e5-9470-6acaf43aabbe nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.152151111 +0000 UTC m=+52.721561863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls") pod "image-registry-6fb86598f-m8z72" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe") : secret "image-registry-tls" not found Jul 14 17:05:42.253251 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:42.253216 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:42.253396 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.253368 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:42.253447 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:42.253439 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls podName:5c3c9438-260b-4e36-91c9-d6d8b300965d nodeName:}" failed. No retries permitted until 2026-07-14 17:05:50.253420454 +0000 UTC m=+52.822831227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-b4wp2" (UID: "5c3c9438-260b-4e36-91c9-d6d8b300965d") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:43.628872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:43.628840 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" event={"ID":"e4845b9f-e512-48e6-b8d4-2ae25652b0d0","Type":"ContainerStarted","Data":"be2132a1b879ce7666187411fd60a01dbb20a544daeb3ae1c93d9b3d851a584a"} Jul 14 17:05:44.634421 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.634381 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" event={"ID":"75d34625-d739-4296-be29-f90c173c1a9c","Type":"ContainerStarted","Data":"601e3adca0dd310214e71ea235670768ca8318f6ad73eee5a9b74b958d9f967b"} Jul 14 17:05:44.636332 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.636299 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c4hcb" event={"ID":"2f955fd3-d01a-4775-8f30-a6d37287c0e0","Type":"ContainerStarted","Data":"6a1a6a0bca5dce7c04c97ff6e78079f04042b5e043febb93c86f2378975b43c3"} Jul 14 17:05:44.637617 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.637593 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" event={"ID":"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98","Type":"ContainerStarted","Data":"2e0814d50f14c70547bd1c4b20e2f92cdcdb759202977fe7fe500b4ca7629abf"} Jul 14 17:05:44.639288 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.639271 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/0.log" Jul 14 17:05:44.639366 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.639305 2558 generic.go:356] "Generic (PLEG): container finished" podID="c2be7952-ab97-43f0-a8b5-1f227c22b10c" containerID="febf4429e4da0bc8a76c4fdb01ca2064fff79e582b9df733affc427a8c94c515" exitCode=255 Jul 14 17:05:44.639417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.639395 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lpj7g" event={"ID":"c2be7952-ab97-43f0-a8b5-1f227c22b10c","Type":"ContainerDied","Data":"febf4429e4da0bc8a76c4fdb01ca2064fff79e582b9df733affc427a8c94c515"} Jul 14 17:05:44.639576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.639555 2558 scope.go:117] "RemoveContainer" containerID="febf4429e4da0bc8a76c4fdb01ca2064fff79e582b9df733affc427a8c94c515" Jul 14 17:05:44.641111 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.641087 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" event={"ID":"22e1d58b-ca34-4279-9ece-d1062a82451c","Type":"ContainerStarted","Data":"d8e0837e840780166790ade3df6eda6ff2acdf56dc776bc1a6ed54b8e02e5b16"} Jul 14 17:05:44.642682 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.642660 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" event={"ID":"df83cbc3-0758-4506-8e21-6641839d5f8b","Type":"ContainerStarted","Data":"234acac67c5d0ed9e9c72b719c425f2dfdee6842d37ea550bcca9d16f0cc4674"} Jul 14 17:05:44.653423 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.653367 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" podStartSLOduration=11.042523843 podStartE2EDuration="18.653349676s" totalImagesPullingTime="7.610825833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:44.652350367 +0000 UTC m=+47.221761159" watchObservedRunningTime="2026-07-14 17:05:44.653349676 +0000 UTC m=+47.222760448" Jul 14 17:05:44.692407 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.692378 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:44.692559 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.692432 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:44.694313 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.694260 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c4hcb" podStartSLOduration=3.116810531 podStartE2EDuration="10.694240647s" totalImagesPullingTime="7.577430116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:44.6933227 +0000 UTC m=+47.262733508" watchObservedRunningTime="2026-07-14 17:05:44.694240647 +0000 UTC m=+47.263651431" Jul 14 17:05:44.695010 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.694965 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g2gjj" podStartSLOduration=14.820453027 podStartE2EDuration="18.694945124s" totalImagesPullingTime="3.874492097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:44.671248844 +0000 UTC m=+47.240659615" watchObservedRunningTime="2026-07-14 17:05:44.694945124 +0000 UTC m=+47.264355895" Jul 14 17:05:44.742424 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.742354 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" podStartSLOduration=11.158318447 podStartE2EDuration="18.742331339s" totalImagesPullingTime="7.584012892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:44.71462165 +0000 UTC m=+47.284032448" watchObservedRunningTime="2026-07-14 17:05:44.742331339 +0000 UTC m=+47.311742109" Jul 14 17:05:44.760218 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.759501 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2mrxl" podStartSLOduration=11.180588314 podStartE2EDuration="18.7594772s" totalImagesPullingTime="7.578888886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:44.758753165 +0000 UTC m=+47.328163962" watchObservedRunningTime="2026-07-14 17:05:44.7594772 +0000 UTC m=+47.328887972" Jul 14 17:05:44.783718 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.783594 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" podStartSLOduration=11.207054892 podStartE2EDuration="18.78357174s" totalImagesPullingTime="7.576516848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:44.783134537 +0000 UTC m=+47.352545309" watchObservedRunningTime="2026-07-14 17:05:44.78357174 +0000 UTC m=+47.352982515" Jul 14 17:05:44.818580 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.817568 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hflz4"] Jul 14 17:05:44.840349 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.838982 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.842656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.840965 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hflz4"] Jul 14 17:05:44.851731 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.851704 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 17:05:44.852375 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.852296 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6x48r\"" Jul 14 17:05:44.852515 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.852488 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 17:05:44.878719 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.878684 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.878889 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.878814 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-data-volume\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.878949 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.878897 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.879002 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.878939 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqh4z\" (UniqueName: \"kubernetes.io/projected/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-api-access-kqh4z\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.879072 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.879057 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-crio-socket\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.980367 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.980333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.980573 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.980397 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-data-volume\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.980573 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.980443 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.980573 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.980460 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqh4z\" (UniqueName: \"kubernetes.io/projected/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-api-access-kqh4z\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.980573 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:44.980480 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:44.980573 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:44.980576 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:45.480552719 +0000 UTC m=+48.049963484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : configmap "kube-rbac-proxy" not found Jul 14 17:05:44.980838 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.980610 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-crio-socket\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.980838 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:44.980617 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:44.980838 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:44.980733 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:45.48071451 +0000 UTC m=+48.050125278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:44.980838 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.980746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-data-volume\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.980838 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.980812 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-crio-socket\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:44.991596 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:44.991565 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqh4z\" (UniqueName: \"kubernetes.io/projected/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-api-access-kqh4z\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:45.485243 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.485208 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:45.485439 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:45.485325 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:45.485439 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.485353 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:45.485439 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:45.485389 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:46.485373381 +0000 UTC m=+49.054784129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:45.485629 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:45.485441 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:45.485629 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:45.485499 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:46.485483788 +0000 UTC m=+49.054894555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : configmap "kube-rbac-proxy" not found Jul 14 17:05:45.648587 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.648533 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:05:45.649024 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.648986 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/0.log" Jul 14 17:05:45.649084 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.649021 2558 generic.go:356] "Generic (PLEG): container finished" podID="c2be7952-ab97-43f0-a8b5-1f227c22b10c" containerID="2c97af692119c757f1cad56aa27928ef5a87d88cb365ec5b072131689f0b2c6d" exitCode=255 Jul 14 17:05:45.649084 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.649054 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lpj7g" event={"ID":"c2be7952-ab97-43f0-a8b5-1f227c22b10c","Type":"ContainerDied","Data":"2c97af692119c757f1cad56aa27928ef5a87d88cb365ec5b072131689f0b2c6d"} Jul 14 17:05:45.649181 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.649120 2558 scope.go:117] "RemoveContainer" containerID="febf4429e4da0bc8a76c4fdb01ca2064fff79e582b9df733affc427a8c94c515" Jul 14 17:05:45.649355 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:45.649330 2558 scope.go:117] "RemoveContainer" containerID="2c97af692119c757f1cad56aa27928ef5a87d88cb365ec5b072131689f0b2c6d" Jul 14 17:05:45.649574 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:45.649528 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-lpj7g_openshift-console-operator(c2be7952-ab97-43f0-a8b5-1f227c22b10c)\"" pod="openshift-console-operator/console-operator-7866497494-lpj7g" podUID="c2be7952-ab97-43f0-a8b5-1f227c22b10c" Jul 14 17:05:46.495829 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:46.495777 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:46.496087 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:46.495945 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:46.496087 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:46.495963 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:46.496087 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:46.496037 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:48.496014703 +0000 UTC m=+51.065425463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:46.496087 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:46.496037 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:46.496309 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:46.496094 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:48.496078291 +0000 UTC m=+51.065489045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : configmap "kube-rbac-proxy" not found Jul 14 17:05:46.654625 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:46.654590 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:05:46.654998 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:46.654965 2558 scope.go:117] "RemoveContainer" containerID="2c97af692119c757f1cad56aa27928ef5a87d88cb365ec5b072131689f0b2c6d" Jul 14 17:05:46.655134 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:46.655115 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-lpj7g_openshift-console-operator(c2be7952-ab97-43f0-a8b5-1f227c22b10c)\"" pod="openshift-console-operator/console-operator-7866497494-lpj7g" podUID="c2be7952-ab97-43f0-a8b5-1f227c22b10c" Jul 14 17:05:46.660315 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:46.660290 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pkfmh_2bc646b1-d627-4d8f-a261-d84275abdb68/dns-node-resolver/0.log" Jul 14 17:05:47.261464 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:47.261431 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2fd85_6992d303-6bd8-485d-b532-f782c6809cbc/node-ca/0.log" Jul 14 17:05:48.060076 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:48.060046 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c4hcb_2f955fd3-d01a-4775-8f30-a6d37287c0e0/serve-healthcheck-canary/0.log" Jul 14 17:05:48.515915 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:48.515876 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:48.516110 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:48.516028 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:48.516110 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:48.516059 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:48.516221 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:48.516123 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:48.516221 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:48.516138 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:52.516120306 +0000 UTC m=+55.085531074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:48.516221 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:48.516179 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:52.516161647 +0000 UTC m=+55.085572401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : configmap "kube-rbac-proxy" not found Jul 14 17:05:50.232941 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:50.232897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:50.232975 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:50.233024 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233072 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233105 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233130 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233152 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb86598f-m8z72: secret "image-registry-tls" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233141 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:06.233121492 +0000 UTC m=+68.802532251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : secret "router-metrics-certs-default" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233206 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls podName:523c6d19-8d88-45d5-9560-8f32e260aa1b nodeName:}" failed. No retries permitted until 2026-07-14 17:06:06.233192133 +0000 UTC m=+68.802602886 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls") pod "cluster-samples-operator-65db858894-z84mm" (UID: "523c6d19-8d88-45d5-9560-8f32e260aa1b") : secret "samples-operator-tls" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:50.233259 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233295 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls podName:bc87436a-95c3-40e5-9470-6acaf43aabbe nodeName:}" failed. No retries permitted until 2026-07-14 17:06:06.233272049 +0000 UTC m=+68.802682803 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls") pod "image-registry-6fb86598f-m8z72" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe") : secret "image-registry-tls" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233312 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233335 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert podName:e812b93e-638c-429e-93cd-c4fa57fc3f73 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:06.233328123 +0000 UTC m=+68.802738871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-bf2qg" (UID: "e812b93e-638c-429e-93cd-c4fa57fc3f73") : secret "networking-console-plugin-cert" not found Jul 14 17:05:50.233389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:50.233329 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:05:50.233899 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:50.233375 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:05:50.233899 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233415 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:50.233899 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233450 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls podName:5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f nodeName:}" failed. No retries permitted until 2026-07-14 17:06:06.233442378 +0000 UTC m=+68.802853127 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls") pod "dns-default-7g9zk" (UID: "5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f") : secret "dns-default-metrics-tls" not found Jul 14 17:05:50.233899 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.233581 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle podName:fef52ed7-0972-4f2e-8cc0-2780b7f59855 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:06.233561945 +0000 UTC m=+68.802972711 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle") pod "router-default-7497bbf74b-2lkjk" (UID: "fef52ed7-0972-4f2e-8cc0-2780b7f59855") : configmap references non-existent config key: service-ca.crt Jul 14 17:05:50.334101 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:50.334065 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:05:50.334279 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.334215 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:50.334325 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:50.334286 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls podName:5c3c9438-260b-4e36-91c9-d6d8b300965d nodeName:}" failed. No retries permitted until 2026-07-14 17:06:06.334270516 +0000 UTC m=+68.903681265 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-b4wp2" (UID: "5c3c9438-260b-4e36-91c9-d6d8b300965d") : secret "cluster-monitoring-operator-tls" not found Jul 14 17:05:52.555858 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:52.555816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:52.556311 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:52.555908 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:52.556311 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:52.556062 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:52.556311 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:52.556131 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls podName:f9eb7757-e64d-4550-bc5f-e5c6841bfb52 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:00.556115577 +0000 UTC m=+63.125526331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hflz4" (UID: "f9eb7757-e64d-4550-bc5f-e5c6841bfb52") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:52.556421 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:52.556362 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:05:54.692674 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:54.692638 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:54.693074 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:54.692697 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:05:54.693074 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:05:54.693033 2558 scope.go:117] "RemoveContainer" containerID="2c97af692119c757f1cad56aa27928ef5a87d88cb365ec5b072131689f0b2c6d" Jul 14 17:05:54.693194 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:05:54.693174 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-lpj7g_openshift-console-operator(c2be7952-ab97-43f0-a8b5-1f227c22b10c)\"" pod="openshift-console-operator/console-operator-7866497494-lpj7g" podUID="c2be7952-ab97-43f0-a8b5-1f227c22b10c" Jul 14 17:06:00.637075 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:00.637036 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:06:00.639525 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:00.639500 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eb7757-e64d-4550-bc5f-e5c6841bfb52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hflz4\" (UID: \"f9eb7757-e64d-4550-bc5f-e5c6841bfb52\") " pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:06:00.788791 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:00.788752 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6x48r\"" Jul 14 17:06:00.796745 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:00.796717 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hflz4" Jul 14 17:06:00.959934 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:00.959870 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hflz4"] Jul 14 17:06:00.963440 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:00.963412 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9eb7757_e64d_4550_bc5f_e5c6841bfb52.slice/crio-7a5eded1117c722c6eb84481873dce2d6b9b995ca5af83748f2f05d63136ceca WatchSource:0}: Error finding container 7a5eded1117c722c6eb84481873dce2d6b9b995ca5af83748f2f05d63136ceca: Status 404 returned error can't find the container with id 7a5eded1117c722c6eb84481873dce2d6b9b995ca5af83748f2f05d63136ceca Jul 14 17:06:01.562718 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:01.562688 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6sjrd" Jul 14 17:06:01.704910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:01.704873 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hflz4" event={"ID":"f9eb7757-e64d-4550-bc5f-e5c6841bfb52","Type":"ContainerStarted","Data":"7a5eded1117c722c6eb84481873dce2d6b9b995ca5af83748f2f05d63136ceca"} Jul 14 17:06:04.879939 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.879900 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:06:04.880321 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.879960 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:06:04.883242 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.883219 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:06:04.884261 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.884237 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:06:04.892819 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.892792 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba8203bc-44c2-4091-ae5d-f0fb93aba210-original-pull-secret\") pod \"global-pull-secret-syncer-s29kb\" (UID: \"ba8203bc-44c2-4091-ae5d-f0fb93aba210\") " pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:06:04.892819 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.892809 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86199c5a-74c9-446b-9aa5-028fb5224e6c-metrics-certs\") pod \"network-metrics-daemon-sr7b5\" (UID: \"86199c5a-74c9-446b-9aa5-028fb5224e6c\") " pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:06:04.980793 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.980745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:06:04.983752 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:04.983721 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dm5h\" (UniqueName: \"kubernetes.io/projected/3300e5a3-e457-4254-81a3-76412fdb24eb-kube-api-access-5dm5h\") pod \"network-check-target-47glh\" (UID: \"3300e5a3-e457-4254-81a3-76412fdb24eb\") " pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:06:05.032118 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.032085 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7wgm\"" Jul 14 17:06:05.039121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.039090 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr7b5" Jul 14 17:06:05.044804 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.044772 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s29kb" Jul 14 17:06:05.055315 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.055277 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tv9jb\"" Jul 14 17:06:05.062868 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.062841 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:06:05.341135 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.341101 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-47glh"] Jul 14 17:06:05.342854 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:05.342819 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3300e5a3_e457_4254_81a3_76412fdb24eb.slice/crio-81bf63088b27f09300291407dddc546e9dd49de2edf0c21ea6be5ab253cb4e7f WatchSource:0}: Error finding container 81bf63088b27f09300291407dddc546e9dd49de2edf0c21ea6be5ab253cb4e7f: Status 404 returned error can't find the container with id 81bf63088b27f09300291407dddc546e9dd49de2edf0c21ea6be5ab253cb4e7f Jul 14 17:06:05.562266 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.562241 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s29kb"] Jul 14 17:06:05.564803 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:05.564768 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8203bc_44c2_4091_ae5d_f0fb93aba210.slice/crio-71f01e27e5e6ff25b55fdfaa47c7aca894b47df1febc1352d222179898a2d181 WatchSource:0}: Error finding container 71f01e27e5e6ff25b55fdfaa47c7aca894b47df1febc1352d222179898a2d181: Status 404 returned error can't find the container with id 71f01e27e5e6ff25b55fdfaa47c7aca894b47df1febc1352d222179898a2d181 Jul 14 17:06:05.565484 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.565460 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sr7b5"] Jul 14 17:06:05.567788 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:05.567766 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86199c5a_74c9_446b_9aa5_028fb5224e6c.slice/crio-4132fc6969193b5be1e3c4c4291011684b5bd7205033150cb3a7935a1d833dab WatchSource:0}: Error finding container 4132fc6969193b5be1e3c4c4291011684b5bd7205033150cb3a7935a1d833dab: Status 404 returned error can't find the container with id 4132fc6969193b5be1e3c4c4291011684b5bd7205033150cb3a7935a1d833dab Jul 14 17:06:05.719381 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.719297 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s29kb" event={"ID":"ba8203bc-44c2-4091-ae5d-f0fb93aba210","Type":"ContainerStarted","Data":"71f01e27e5e6ff25b55fdfaa47c7aca894b47df1febc1352d222179898a2d181"} Jul 14 17:06:05.721112 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.721078 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47glh" event={"ID":"3300e5a3-e457-4254-81a3-76412fdb24eb","Type":"ContainerStarted","Data":"5334da58bec99374045b4f75fd2cf57734b8d1c409e6f122e17182054d8ef6c1"} Jul 14 17:06:05.721286 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.721132 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-47glh" event={"ID":"3300e5a3-e457-4254-81a3-76412fdb24eb","Type":"ContainerStarted","Data":"81bf63088b27f09300291407dddc546e9dd49de2edf0c21ea6be5ab253cb4e7f"} Jul 14 17:06:05.721286 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.721160 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:06:05.722291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.722269 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sr7b5" event={"ID":"86199c5a-74c9-446b-9aa5-028fb5224e6c","Type":"ContainerStarted","Data":"4132fc6969193b5be1e3c4c4291011684b5bd7205033150cb3a7935a1d833dab"} Jul 14 17:06:05.723919 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:05.723885 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hflz4" event={"ID":"f9eb7757-e64d-4550-bc5f-e5c6841bfb52","Type":"ContainerStarted","Data":"094e3b8b83b99110df2f09368a6989449f10ee530d1b8267a48872ba16f59c8d"} Jul 14 17:06:06.295308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.294943 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:06:06.295308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.295012 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:06:06.295308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.295067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:06:06.295308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.295092 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:06:06.295308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.295136 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:06.295308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.295207 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:06.296068 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.296038 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fef52ed7-0972-4f2e-8cc0-2780b7f59855-service-ca-bundle\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:06.298250 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.298222 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f-metrics-tls\") pod \"dns-default-7g9zk\" (UID: \"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f\") " pod="openshift-dns/dns-default-7g9zk" Jul 14 17:06:06.298388 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.298276 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"image-registry-6fb86598f-m8z72\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:06:06.298653 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.298607 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef52ed7-0972-4f2e-8cc0-2780b7f59855-metrics-certs\") pod \"router-default-7497bbf74b-2lkjk\" (UID: \"fef52ed7-0972-4f2e-8cc0-2780b7f59855\") " pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:06.299374 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.299348 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e812b93e-638c-429e-93cd-c4fa57fc3f73-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-bf2qg\" (UID: \"e812b93e-638c-429e-93cd-c4fa57fc3f73\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:06:06.300003 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.299978 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/523c6d19-8d88-45d5-9560-8f32e260aa1b-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-z84mm\" (UID: \"523c6d19-8d88-45d5-9560-8f32e260aa1b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:06:06.396322 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.396279 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:06:06.399068 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.399041 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3c9438-260b-4e36-91c9-d6d8b300965d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-b4wp2\" (UID: \"5c3c9438-260b-4e36-91c9-d6d8b300965d\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:06:06.440976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.440938 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-888d9\"" Jul 14 17:06:06.448447 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.448418 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:06:06.475042 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.475014 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8v8vb\"" Jul 14 17:06:06.482583 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.482534 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7g9zk" Jul 14 17:06:06.488588 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.488554 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fssx8\"" Jul 14 17:06:06.496024 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.495968 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" Jul 14 17:06:06.508994 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.508971 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cbcjd\"" Jul 14 17:06:06.516927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.516895 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" Jul 14 17:06:06.540818 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.540168 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bnmjj\"" Jul 14 17:06:06.547280 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.547205 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:06.568616 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.568553 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4z8fb\"" Jul 14 17:06:06.570844 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.570534 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" Jul 14 17:06:06.685101 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.684452 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-47glh" podStartSLOduration=66.684427383 podStartE2EDuration="1m6.684427383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:05.739594069 +0000 UTC m=+68.309004862" watchObservedRunningTime="2026-07-14 17:06:06.684427383 +0000 UTC m=+69.253838155" Jul 14 17:06:06.686839 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.686605 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb86598f-m8z72"] Jul 14 17:06:06.695164 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:06.694445 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc87436a_95c3_40e5_9470_6acaf43aabbe.slice/crio-0332bcf31085a42afecc5f17f610f38bdd78aa38dbca02637c3ccf2dbb838e47 WatchSource:0}: Error finding container 0332bcf31085a42afecc5f17f610f38bdd78aa38dbca02637c3ccf2dbb838e47: Status 404 returned error can't find the container with id 0332bcf31085a42afecc5f17f610f38bdd78aa38dbca02637c3ccf2dbb838e47 Jul 14 17:06:06.709589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.709339 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7g9zk"] Jul 14 17:06:06.734996 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.734862 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" event={"ID":"bc87436a-95c3-40e5-9470-6acaf43aabbe","Type":"ContainerStarted","Data":"0332bcf31085a42afecc5f17f610f38bdd78aa38dbca02637c3ccf2dbb838e47"} Jul 14 17:06:06.737386 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.737358 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hflz4" event={"ID":"f9eb7757-e64d-4550-bc5f-e5c6841bfb52","Type":"ContainerStarted","Data":"fea5032dd03e3e0be561830819c5c5cd40b1dd8130697ff9e21e17bc414e6b21"} Jul 14 17:06:06.739798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.739728 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7g9zk" event={"ID":"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f","Type":"ContainerStarted","Data":"933ece2633d1eb96eb8a6f5c57164d43f1ab54c5ded5fd57b4f3a99018a908ec"} Jul 14 17:06:06.798635 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.796854 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-bf2qg"] Jul 14 17:06:06.804365 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:06.804319 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode812b93e_638c_429e_93cd_c4fa57fc3f73.slice/crio-62fce1ec3c98e88204857cc9dac126b7e7805f84492c76e8ce5afa88e8154bde WatchSource:0}: Error finding container 62fce1ec3c98e88204857cc9dac126b7e7805f84492c76e8ce5afa88e8154bde: Status 404 returned error can't find the container with id 62fce1ec3c98e88204857cc9dac126b7e7805f84492c76e8ce5afa88e8154bde Jul 14 17:06:06.821986 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.821918 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2"] Jul 14 17:06:06.823120 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.823050 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7497bbf74b-2lkjk"] Jul 14 17:06:06.927480 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:06.927363 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm"] Jul 14 17:06:07.376352 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.376325 2558 scope.go:117] "RemoveContainer" containerID="2c97af692119c757f1cad56aa27928ef5a87d88cb365ec5b072131689f0b2c6d" Jul 14 17:06:07.747275 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.746565 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" event={"ID":"fef52ed7-0972-4f2e-8cc0-2780b7f59855","Type":"ContainerStarted","Data":"b7a922709017d0823a3f622a9eacae223b7f0ef4359c138b38b45dd922310714"} Jul 14 17:06:07.747275 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.746631 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" event={"ID":"fef52ed7-0972-4f2e-8cc0-2780b7f59855","Type":"ContainerStarted","Data":"d82a4c0baff540cbd122ce2a78038e88e1d5c20998c196cc7851a296e87a767e"} Jul 14 17:06:07.750966 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.750937 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:06:07.751121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.751034 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lpj7g" event={"ID":"c2be7952-ab97-43f0-a8b5-1f227c22b10c","Type":"ContainerStarted","Data":"474a36ae5961b5a8d43d931f69308afd12d910d92acb8354418c05360a4d36f2"} Jul 14 17:06:07.754330 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.754303 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:06:07.758352 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.757722 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" event={"ID":"bc87436a-95c3-40e5-9470-6acaf43aabbe","Type":"ContainerStarted","Data":"bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f"} Jul 14 17:06:07.758352 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.758309 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:06:07.761140 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.761084 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" event={"ID":"523c6d19-8d88-45d5-9560-8f32e260aa1b","Type":"ContainerStarted","Data":"d880732322f37ad633881ddf60d4a6a7d264efcc8e8df88edd156b7b13c260c8"} Jul 14 17:06:07.767065 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.767037 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sr7b5" event={"ID":"86199c5a-74c9-446b-9aa5-028fb5224e6c","Type":"ContainerStarted","Data":"b21a5010d74f539a8d23957cbaf645cec0a6c72823ebfd05e24d34ef36767730"} Jul 14 17:06:07.767191 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.767088 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sr7b5" event={"ID":"86199c5a-74c9-446b-9aa5-028fb5224e6c","Type":"ContainerStarted","Data":"c3b046822e41284b93241ceb98dc81b892fb4a48b74864c7c0c42ad4c1156023"} Jul 14 17:06:07.768780 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.768748 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" event={"ID":"e812b93e-638c-429e-93cd-c4fa57fc3f73","Type":"ContainerStarted","Data":"62fce1ec3c98e88204857cc9dac126b7e7805f84492c76e8ce5afa88e8154bde"} Jul 14 17:06:07.770722 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.770675 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" event={"ID":"5c3c9438-260b-4e36-91c9-d6d8b300965d","Type":"ContainerStarted","Data":"53356427788ed4e6ac294a0ac3eae28bf6d16b8fdb117e5a00c1b0f45956e72e"} Jul 14 17:06:07.776470 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.775320 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" podStartSLOduration=41.7753018 podStartE2EDuration="41.7753018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:07.774712388 +0000 UTC m=+70.344123162" watchObservedRunningTime="2026-07-14 17:06:07.7753018 +0000 UTC m=+70.344712574" Jul 14 17:06:07.809749 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.809454 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" podStartSLOduration=67.809433125 podStartE2EDuration="1m7.809433125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:07.807009343 +0000 UTC m=+70.376420139" watchObservedRunningTime="2026-07-14 17:06:07.809433125 +0000 UTC m=+70.378843894" Jul 14 17:06:07.890496 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.889744 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sr7b5" podStartSLOduration=67.871095657 podStartE2EDuration="1m8.889717143s" totalImagesPullingTime="1.018621486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:07.88905939 +0000 UTC m=+70.458470162" watchObservedRunningTime="2026-07-14 17:06:07.889717143 +0000 UTC m=+70.459127915" Jul 14 17:06:07.890496 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:07.890094 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-lpj7g" podStartSLOduration=34.311881772 podStartE2EDuration="41.89008538s" totalImagesPullingTime="7.578203608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:07.843230925 +0000 UTC m=+70.412641699" watchObservedRunningTime="2026-07-14 17:06:07.89008538 +0000 UTC m=+70.459496153" Jul 14 17:06:08.458502 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:08.458465 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-lpj7g" Jul 14 17:06:08.547834 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:08.547804 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:08.551215 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:08.550993 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:08.778623 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:08.778558 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:08.780285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:08.780081 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7497bbf74b-2lkjk" Jul 14 17:06:15.464337 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.464255 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls"] Jul 14 17:06:15.478775 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.478743 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls"] Jul 14 17:06:15.478933 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.478863 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:15.481775 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.481749 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2z5sg\"" Jul 14 17:06:15.482264 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.482247 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 14 17:06:15.569520 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.569479 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7d22368a-6361-4381-9a34-ca1aca90a464-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bb9ls\" (UID: \"7d22368a-6361-4381-9a34-ca1aca90a464\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:15.670587 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.670549 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7d22368a-6361-4381-9a34-ca1aca90a464-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bb9ls\" (UID: \"7d22368a-6361-4381-9a34-ca1aca90a464\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:15.670731 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:06:15.670694 2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 14 17:06:15.670771 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:06:15.670761 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d22368a-6361-4381-9a34-ca1aca90a464-tls-certificates podName:7d22368a-6361-4381-9a34-ca1aca90a464 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:16.170743156 +0000 UTC m=+78.740153904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/7d22368a-6361-4381-9a34-ca1aca90a464-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-bb9ls" (UID: "7d22368a-6361-4381-9a34-ca1aca90a464") : secret "prometheus-operator-admission-webhook-tls" not found Jul 14 17:06:15.804868 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.804831 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s29kb" event={"ID":"ba8203bc-44c2-4091-ae5d-f0fb93aba210","Type":"ContainerStarted","Data":"12bfd7734a929183da2342222d82cf5f68619031719b53a85ebd102482c7e46d"} Jul 14 17:06:15.806749 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.806720 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7g9zk" event={"ID":"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f","Type":"ContainerStarted","Data":"d15d957b30549032518f7cc9fea32c92921f0b00882801c64d5a2784a6099d39"} Jul 14 17:06:15.806913 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.806779 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7g9zk" Jul 14 17:06:15.806913 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.806806 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7g9zk" event={"ID":"5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f","Type":"ContainerStarted","Data":"30329292817a7f97e9d2175768d3e3c881094e5897082687ff9cfe184272e96c"} Jul 14 17:06:15.808523 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.808496 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" event={"ID":"523c6d19-8d88-45d5-9560-8f32e260aa1b","Type":"ContainerStarted","Data":"5dfc4aae996f0991c6f4c00d92a2412f0cd8c4bde6dcb1248756104104ab284a"} Jul 14 17:06:15.808691 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.808557 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" event={"ID":"523c6d19-8d88-45d5-9560-8f32e260aa1b","Type":"ContainerStarted","Data":"f0a6d2fb90bd0b5ea8588e9281be5761d1e7a8bf685d5173dbaa96937de0df26"} Jul 14 17:06:15.810030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.810007 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" event={"ID":"e812b93e-638c-429e-93cd-c4fa57fc3f73","Type":"ContainerStarted","Data":"9429fcec80b65c3ec440a2334d49f56f4ae4f8821cd68e88e7ddc90274e04408"} Jul 14 17:06:15.811933 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.811911 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hflz4" event={"ID":"f9eb7757-e64d-4550-bc5f-e5c6841bfb52","Type":"ContainerStarted","Data":"c1080593c966c8bdf0300d2f6f05c6dff97fa615b771eb1e683757e721cfb802"} Jul 14 17:06:15.813182 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.813161 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" event={"ID":"5c3c9438-260b-4e36-91c9-d6d8b300965d","Type":"ContainerStarted","Data":"517723a0afc21b98eedebb960ed87a97988502404283753190005c99eb74af14"} Jul 14 17:06:15.823764 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.823722 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s29kb" podStartSLOduration=66.630900602 podStartE2EDuration="1m15.823708473s" totalImagesPullingTime="9.192807871s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.822814811 +0000 UTC m=+78.392225581" watchObservedRunningTime="2026-07-14 17:06:15.823708473 +0000 UTC m=+78.393119243" Jul 14 17:06:15.845307 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.845223 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hflz4" podStartSLOduration=18.192996617 podStartE2EDuration="31.84520857s" totalImagesPullingTime="13.652211953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.844357291 +0000 UTC m=+78.413768061" watchObservedRunningTime="2026-07-14 17:06:15.84520857 +0000 UTC m=+78.414619340" Jul 14 17:06:15.865291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.865233 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-b4wp2" podStartSLOduration=41.936223555 podStartE2EDuration="49.865215877s" totalImagesPullingTime="7.928992322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.864580231 +0000 UTC m=+78.433991033" watchObservedRunningTime="2026-07-14 17:06:15.865215877 +0000 UTC m=+78.434626641" Jul 14 17:06:15.898119 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.898038 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-z84mm" podStartSLOduration=42.11678162 podStartE2EDuration="49.89802145s" totalImagesPullingTime="7.78123983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.897096374 +0000 UTC m=+78.466507157" watchObservedRunningTime="2026-07-14 17:06:15.89802145 +0000 UTC m=+78.467432221" Jul 14 17:06:15.917605 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.917530 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-bf2qg" podStartSLOduration=41.967162123 podStartE2EDuration="49.917513256s" totalImagesPullingTime="7.950351133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.917235071 +0000 UTC m=+78.486645847" watchObservedRunningTime="2026-07-14 17:06:15.917513256 +0000 UTC m=+78.486924026" Jul 14 17:06:15.941531 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:15.941472 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7g9zk" podStartSLOduration=33.908137247 podStartE2EDuration="41.941454991s" totalImagesPullingTime="8.033317744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:15.940837408 +0000 UTC m=+78.510248179" watchObservedRunningTime="2026-07-14 17:06:15.941454991 +0000 UTC m=+78.510865761" Jul 14 17:06:16.175690 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:16.175587 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7d22368a-6361-4381-9a34-ca1aca90a464-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bb9ls\" (UID: \"7d22368a-6361-4381-9a34-ca1aca90a464\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:16.178110 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:16.178083 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7d22368a-6361-4381-9a34-ca1aca90a464-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bb9ls\" (UID: \"7d22368a-6361-4381-9a34-ca1aca90a464\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:16.390900 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:16.390858 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:16.529223 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:16.529197 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls"] Jul 14 17:06:16.531534 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:16.531488 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d22368a_6361_4381_9a34_ca1aca90a464.slice/crio-aa0e8406acc29c0f5dbb70349cf64db5ab48e5d55959cc5a5f4a7f573f42e44f WatchSource:0}: Error finding container aa0e8406acc29c0f5dbb70349cf64db5ab48e5d55959cc5a5f4a7f573f42e44f: Status 404 returned error can't find the container with id aa0e8406acc29c0f5dbb70349cf64db5ab48e5d55959cc5a5f4a7f573f42e44f Jul 14 17:06:16.818779 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:16.818739 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" event={"ID":"7d22368a-6361-4381-9a34-ca1aca90a464","Type":"ContainerStarted","Data":"aa0e8406acc29c0f5dbb70349cf64db5ab48e5d55959cc5a5f4a7f573f42e44f"} Jul 14 17:06:18.827860 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:18.827822 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" event={"ID":"7d22368a-6361-4381-9a34-ca1aca90a464","Type":"ContainerStarted","Data":"1895e13ab2ec62a67a6ff38b6baae008ef240cb0a3d34b3cfa6f9fe70b1afd87"} Jul 14 17:06:18.828297 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:18.828032 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:18.833043 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:18.833020 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" Jul 14 17:06:18.846378 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:18.846329 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bb9ls" podStartSLOduration=2.243230875 podStartE2EDuration="3.846313528s" totalImagesPullingTime="1.603082653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:18.844900732 +0000 UTC m=+81.414311529" watchObservedRunningTime="2026-07-14 17:06:18.846313528 +0000 UTC m=+81.415724299" Jul 14 17:06:25.821005 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:25.820967 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7g9zk" Jul 14 17:06:25.989173 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:25.989133 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc"] Jul 14 17:06:25.992480 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:25.992458 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.007935 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.007906 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 14 17:06:26.008253 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.008232 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bqmfn\"" Jul 14 17:06:26.012581 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.012563 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 14 17:06:26.012715 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.012700 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 17:06:26.041078 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.041047 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc"] Jul 14 17:06:26.058168 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.058135 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe093afe-a132-497b-815b-229484d06d6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.058337 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.058192 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe093afe-a132-497b-815b-229484d06d6a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.058337 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.058261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe093afe-a132-497b-815b-229484d06d6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.058337 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.058303 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mmpg\" (UniqueName: \"kubernetes.io/projected/fe093afe-a132-497b-815b-229484d06d6a-kube-api-access-8mmpg\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.063428 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.063402 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w8vj5"] Jul 14 17:06:26.066001 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.065980 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.070625 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.069751 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 17:06:26.071056 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.070972 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pxj4z\"" Jul 14 17:06:26.071606 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.071585 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 17:06:26.077869 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.077849 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 17:06:26.112909 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.112879 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-rqqlg"] Jul 14 17:06:26.115861 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.115838 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.120463 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.120441 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 14 17:06:26.120867 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.120837 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 14 17:06:26.120867 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.120858 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rdq4r\"" Jul 14 17:06:26.121022 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.120986 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 14 17:06:26.133139 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.133107 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-rqqlg"] Jul 14 17:06:26.159020 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.158992 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d947656e-0a97-44b0-9218-5f36dbc3d5cb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.159194 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe093afe-a132-497b-815b-229484d06d6a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.159194 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159048 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-sys\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159194 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159075 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159194 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159164 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe093afe-a132-497b-815b-229484d06d6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.159417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159203 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56p62\" (UniqueName: \"kubernetes.io/projected/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-api-access-56p62\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.159417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.159417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-textfile\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159326 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mmpg\" (UniqueName: \"kubernetes.io/projected/fe093afe-a132-497b-815b-229484d06d6a-kube-api-access-8mmpg\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.159417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159343 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghfsl\" (UniqueName: \"kubernetes.io/projected/e6ddef09-af7f-4493-a6dc-809f0b4e7190-kube-api-access-ghfsl\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159380 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-wtmp\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159417 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe093afe-a132-497b-815b-229484d06d6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.159872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159436 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-tls\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159458 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159483 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d947656e-0a97-44b0-9218-5f36dbc3d5cb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.159872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159508 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-root\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.159872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159571 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.159872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159595 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.159872 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159619 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6ddef09-af7f-4493-a6dc-809f0b4e7190-metrics-client-ca\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.160108 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.159887 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe093afe-a132-497b-815b-229484d06d6a-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.161662 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.161642 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe093afe-a132-497b-815b-229484d06d6a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.161896 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.161877 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe093afe-a132-497b-815b-229484d06d6a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.188634 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.188601 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mmpg\" (UniqueName: \"kubernetes.io/projected/fe093afe-a132-497b-815b-229484d06d6a-kube-api-access-8mmpg\") pod \"openshift-state-metrics-84bd68b896-t4hpc\" (UID: \"fe093afe-a132-497b-815b-229484d06d6a\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.260706 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.260706 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260710 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56p62\" (UniqueName: \"kubernetes.io/projected/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-api-access-56p62\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260733 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260767 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-textfile\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260785 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghfsl\" (UniqueName: \"kubernetes.io/projected/e6ddef09-af7f-4493-a6dc-809f0b4e7190-kube-api-access-ghfsl\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-wtmp\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260823 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-tls\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260838 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260862 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d947656e-0a97-44b0-9218-5f36dbc3d5cb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260885 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-root\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260916 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260939 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.260964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260965 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6ddef09-af7f-4493-a6dc-809f0b4e7190-metrics-client-ca\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.260998 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d947656e-0a97-44b0-9218-5f36dbc3d5cb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261004 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-wtmp\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-sys\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:06:26.261123 2558 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:06:26.261203 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-tls podName:d947656e-0a97-44b0-9218-5f36dbc3d5cb nodeName:}" failed. No retries permitted until 2026-07-14 17:06:26.76118249 +0000 UTC m=+89.330593240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-rqqlg" (UID: "d947656e-0a97-44b0-9218-5f36dbc3d5cb") : secret "kube-state-metrics-tls" not found Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261269 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-textfile\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261355 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-root\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.261579 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261450 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6ddef09-af7f-4493-a6dc-809f0b4e7190-sys\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.261936 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261637 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-accelerators-collector-config\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.261936 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261766 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.261936 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.261775 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d947656e-0a97-44b0-9218-5f36dbc3d5cb-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.262073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.262051 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6ddef09-af7f-4493-a6dc-809f0b4e7190-metrics-client-ca\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.262306 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.262284 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d947656e-0a97-44b0-9218-5f36dbc3d5cb-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.263785 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.263758 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-tls\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.263876 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.263769 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6ddef09-af7f-4493-a6dc-809f0b4e7190-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.263876 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.263822 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.273628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.273597 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56p62\" (UniqueName: \"kubernetes.io/projected/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-api-access-56p62\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.277945 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.277916 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghfsl\" (UniqueName: \"kubernetes.io/projected/e6ddef09-af7f-4493-a6dc-809f0b4e7190-kube-api-access-ghfsl\") pod \"node-exporter-w8vj5\" (UID: \"e6ddef09-af7f-4493-a6dc-809f0b4e7190\") " pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.301736 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.301702 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" Jul 14 17:06:26.380350 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.380317 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w8vj5" Jul 14 17:06:26.395274 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:26.395219 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ddef09_af7f_4493_a6dc_809f0b4e7190.slice/crio-d5f5dab351915a57b561dcde60abfb184ea7bcc524092328c7151fdd0feb9e9b WatchSource:0}: Error finding container d5f5dab351915a57b561dcde60abfb184ea7bcc524092328c7151fdd0feb9e9b: Status 404 returned error can't find the container with id d5f5dab351915a57b561dcde60abfb184ea7bcc524092328c7151fdd0feb9e9b Jul 14 17:06:26.451140 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.451110 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc"] Jul 14 17:06:26.454913 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:26.454883 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe093afe_a132_497b_815b_229484d06d6a.slice/crio-1be6f2709e0ddbcfd8f6e3e53f6c996493e75893b0c9d7746d5d3ead5094d70a WatchSource:0}: Error finding container 1be6f2709e0ddbcfd8f6e3e53f6c996493e75893b0c9d7746d5d3ead5094d70a: Status 404 returned error can't find the container with id 1be6f2709e0ddbcfd8f6e3e53f6c996493e75893b0c9d7746d5d3ead5094d70a Jul 14 17:06:26.764731 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.764700 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.767148 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.767121 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d947656e-0a97-44b0-9218-5f36dbc3d5cb-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-rqqlg\" (UID: \"d947656e-0a97-44b0-9218-5f36dbc3d5cb\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:26.857670 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.857626 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8vj5" event={"ID":"e6ddef09-af7f-4493-a6dc-809f0b4e7190","Type":"ContainerStarted","Data":"d5f5dab351915a57b561dcde60abfb184ea7bcc524092328c7151fdd0feb9e9b"} Jul 14 17:06:26.859608 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.859579 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" event={"ID":"fe093afe-a132-497b-815b-229484d06d6a","Type":"ContainerStarted","Data":"976044fc42c09eac5cf5fd051ea09a5d4d4242fc52fd59dfcae3d8a68e9a3da2"} Jul 14 17:06:26.859738 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.859640 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" event={"ID":"fe093afe-a132-497b-815b-229484d06d6a","Type":"ContainerStarted","Data":"4f09484aaf7a77688b6955703445ce0d9241cfe99b16f35beea1f107265aac2f"} Jul 14 17:06:26.859738 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:26.859668 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" event={"ID":"fe093afe-a132-497b-815b-229484d06d6a","Type":"ContainerStarted","Data":"1be6f2709e0ddbcfd8f6e3e53f6c996493e75893b0c9d7746d5d3ead5094d70a"} Jul 14 17:06:27.025634 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.025523 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" Jul 14 17:06:27.065305 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.065093 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:06:27.068410 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.068386 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.074235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074130 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 17:06:27.074405 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074321 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 17:06:27.074659 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074476 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 17:06:27.074659 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074573 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 17:06:27.074990 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074698 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 17:06:27.074990 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074824 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rllwh\"" Jul 14 17:06:27.074990 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074831 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 17:06:27.074990 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.074892 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 17:06:27.075220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.075017 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 17:06:27.075220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.075137 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 17:06:27.087803 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.087125 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:06:27.169472 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169256 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-config-volume\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169472 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169337 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169472 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169381 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169472 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-config-out\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169470 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169527 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169588 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-web-config\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169649 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169680 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169711 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169748 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169777 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4jf9\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-kube-api-access-d4jf9\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.169820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.169808 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.171149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.171060 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-rqqlg"] Jul 14 17:06:27.173843 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:27.173822 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd947656e_0a97_44b0_9218_5f36dbc3d5cb.slice/crio-8aefd258ad4f02c64c02ec12efdc5625a0b4a797ecac4f1cc6ae38fd5779297d WatchSource:0}: Error finding container 8aefd258ad4f02c64c02ec12efdc5625a0b4a797ecac4f1cc6ae38fd5779297d: Status 404 returned error can't find the container with id 8aefd258ad4f02c64c02ec12efdc5625a0b4a797ecac4f1cc6ae38fd5779297d Jul 14 17:06:27.270598 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270559 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.270802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270617 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-web-config\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.270802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270665 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.270802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.270802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270710 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.270802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.270802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270767 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4jf9\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-kube-api-access-d4jf9\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.270802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.270797 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.271161 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.271077 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-config-volume\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.271219 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.271157 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.271219 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.271190 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.271331 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.271228 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-config-out\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.271331 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.271261 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.271432 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.271389 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.272913 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.272239 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.272913 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.272863 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.275927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.275839 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-web-config\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.277073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.276351 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.277073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.276681 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.277073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.276755 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-config-volume\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.277073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.276762 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.277073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.277029 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.277449 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.277197 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.277449 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.277213 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.278060 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.278035 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-config-out\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.282678 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.282655 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4jf9\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-kube-api-access-d4jf9\") pod \"alertmanager-main-0\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.382867 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.382832 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:06:27.563763 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.563727 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:06:27.566809 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:27.566770 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7903c5_3b96_4915_929e_377f4e8da338.slice/crio-37bc3408de54c9f9552f22b795d2629e427f2ab976e751c8f14ce37f3cead8f2 WatchSource:0}: Error finding container 37bc3408de54c9f9552f22b795d2629e427f2ab976e751c8f14ce37f3cead8f2: Status 404 returned error can't find the container with id 37bc3408de54c9f9552f22b795d2629e427f2ab976e751c8f14ce37f3cead8f2 Jul 14 17:06:27.865217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.865115 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" event={"ID":"d947656e-0a97-44b0-9218-5f36dbc3d5cb","Type":"ContainerStarted","Data":"8aefd258ad4f02c64c02ec12efdc5625a0b4a797ecac4f1cc6ae38fd5779297d"} Jul 14 17:06:27.866585 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:27.866533 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerStarted","Data":"37bc3408de54c9f9552f22b795d2629e427f2ab976e751c8f14ce37f3cead8f2"} Jul 14 17:06:28.053146 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.053109 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc"] Jul 14 17:06:28.057410 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.057380 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.060363 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.060340 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 14 17:06:28.060363 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.060343 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 14 17:06:28.060574 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.060391 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 14 17:06:28.060574 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.060407 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 14 17:06:28.060574 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.060423 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6547kdlsmml3i\"" Jul 14 17:06:28.060574 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.060363 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 14 17:06:28.060746 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.060595 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2z9x8\"" Jul 14 17:06:28.073964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.073938 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc"] Jul 14 17:06:28.180936 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.180912 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.181091 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.180952 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-tls\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.181091 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.181017 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc9d977a-1414-4de2-97eb-cf9368d51887-metrics-client-ca\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.181091 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.181055 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4hl5\" (UniqueName: \"kubernetes.io/projected/fc9d977a-1414-4de2-97eb-cf9368d51887-kube-api-access-d4hl5\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.181091 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.181082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.181238 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.181121 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.181238 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.181146 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.181238 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.181166 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-grpc-tls\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282259 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282091 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282259 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282170 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-grpc-tls\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282259 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282247 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282432 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282287 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-tls\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282432 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282324 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc9d977a-1414-4de2-97eb-cf9368d51887-metrics-client-ca\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282530 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282493 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4hl5\" (UniqueName: \"kubernetes.io/projected/fc9d977a-1414-4de2-97eb-cf9368d51887-kube-api-access-d4hl5\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.282618 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.282570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.283206 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.283128 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc9d977a-1414-4de2-97eb-cf9368d51887-metrics-client-ca\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.285250 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.285199 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.285475 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.285401 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-tls\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.285614 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.285475 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.285683 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.285615 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-grpc-tls\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.285932 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.285888 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.285932 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.285898 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fc9d977a-1414-4de2-97eb-cf9368d51887-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.297647 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.297589 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4hl5\" (UniqueName: \"kubernetes.io/projected/fc9d977a-1414-4de2-97eb-cf9368d51887-kube-api-access-d4hl5\") pod \"thanos-querier-5bf6fdfc65-rr5pc\" (UID: \"fc9d977a-1414-4de2-97eb-cf9368d51887\") " pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.370703 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.370442 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:28.536077 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.536053 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc"] Jul 14 17:06:28.537423 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:28.537390 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9d977a_1414_4de2_97eb_cf9368d51887.slice/crio-27a2da5cc6dc1fff69bb441bd47c1eca27676961112a2a2a7d68c03024ec9cd3 WatchSource:0}: Error finding container 27a2da5cc6dc1fff69bb441bd47c1eca27676961112a2a2a7d68c03024ec9cd3: Status 404 returned error can't find the container with id 27a2da5cc6dc1fff69bb441bd47c1eca27676961112a2a2a7d68c03024ec9cd3 Jul 14 17:06:28.874687 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.874639 2558 generic.go:356] "Generic (PLEG): container finished" podID="e6ddef09-af7f-4493-a6dc-809f0b4e7190" containerID="08181eb7758c8a6f5e83ad72b027e31deb605c79ab288183d4e6b2c31021e782" exitCode=0 Jul 14 17:06:28.875169 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.874725 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8vj5" event={"ID":"e6ddef09-af7f-4493-a6dc-809f0b4e7190","Type":"ContainerDied","Data":"08181eb7758c8a6f5e83ad72b027e31deb605c79ab288183d4e6b2c31021e782"} Jul 14 17:06:28.877450 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.877415 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" event={"ID":"fe093afe-a132-497b-815b-229484d06d6a","Type":"ContainerStarted","Data":"9d5ab9c6623ca09bef00371597c017cc4cdd1cfe838f929c2cfb5690575ea63c"} Jul 14 17:06:28.879133 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:28.879105 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" event={"ID":"fc9d977a-1414-4de2-97eb-cf9368d51887","Type":"ContainerStarted","Data":"27a2da5cc6dc1fff69bb441bd47c1eca27676961112a2a2a7d68c03024ec9cd3"} Jul 14 17:06:29.787848 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.787254 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:06:29.817575 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.817497 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-t4hpc" podStartSLOduration=3.264698166 podStartE2EDuration="4.817473338s" totalImagesPullingTime="1.552775172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:28.917769766 +0000 UTC m=+91.487180539" watchObservedRunningTime="2026-07-14 17:06:29.817473338 +0000 UTC m=+92.386884114" Jul 14 17:06:29.886435 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.886397 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7903c5-3b96-4915-929e-377f4e8da338" containerID="e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5" exitCode=0 Jul 14 17:06:29.886894 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.886484 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5"} Jul 14 17:06:29.890809 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.890739 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8vj5" event={"ID":"e6ddef09-af7f-4493-a6dc-809f0b4e7190","Type":"ContainerStarted","Data":"57f3ed1d847d65cd71ff40f305a6e1254e1a9633c87fbfe2899d4f2e56b85b4b"} Jul 14 17:06:29.890928 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.890834 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w8vj5" event={"ID":"e6ddef09-af7f-4493-a6dc-809f0b4e7190","Type":"ContainerStarted","Data":"ea9f29a55ddc9ce55a0d4f42ba69c82062365335a7dd3f9ff1291ee2cb50b87b"} Jul 14 17:06:29.895968 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.895940 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" event={"ID":"d947656e-0a97-44b0-9218-5f36dbc3d5cb","Type":"ContainerStarted","Data":"2134ff9c3c883110cea30f5cba93c52a86d5a75e81766433541cfd3cdd323999"} Jul 14 17:06:29.896088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.895996 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" event={"ID":"d947656e-0a97-44b0-9218-5f36dbc3d5cb","Type":"ContainerStarted","Data":"edfaf099d9b79fedc824af15de522cbec4b0eff6b0d563cbe778e6d8a660b82f"} Jul 14 17:06:29.896088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.896023 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" event={"ID":"d947656e-0a97-44b0-9218-5f36dbc3d5cb","Type":"ContainerStarted","Data":"8a2eecc8ddb3b374b901c5d36df1841a9da844adb89c6310b54974e6c6975505"} Jul 14 17:06:29.978416 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:29.978346 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-rqqlg" podStartSLOduration=1.51294227 podStartE2EDuration="3.978329135s" totalImagesPullingTime="2.465386865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:29.977221385 +0000 UTC m=+92.546632180" watchObservedRunningTime="2026-07-14 17:06:29.978329135 +0000 UTC m=+92.547739905" Jul 14 17:06:30.019287 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.019208 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w8vj5" podStartSLOduration=2.200554815 podStartE2EDuration="4.019186244s" totalImagesPullingTime="1.747460625s" totalInitContainerRuntime="71.170804ms" isStatefulPod=true podCreationTimestamp="2026-07-14 17:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:30.019150194 +0000 UTC m=+92.588560965" watchObservedRunningTime="2026-07-14 17:06:30.019186244 +0000 UTC m=+92.588597015" Jul 14 17:06:30.124088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.124049 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fb86598f-m8z72"] Jul 14 17:06:30.691353 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.691318 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw"] Jul 14 17:06:30.694358 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.694334 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:30.697164 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.697140 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 14 17:06:30.697298 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.697219 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-phw5w\"" Jul 14 17:06:30.707009 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.706944 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw"] Jul 14 17:06:30.809443 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.809395 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df56e700-da50-4f28-8718-6529646955ec-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-v8jhw\" (UID: \"df56e700-da50-4f28-8718-6529646955ec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:30.809658 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.809632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmmd7\" (UniqueName: \"kubernetes.io/projected/df56e700-da50-4f28-8718-6529646955ec-kube-api-access-fmmd7\") pod \"monitoring-plugin-5f68df7f58-v8jhw\" (UID: \"df56e700-da50-4f28-8718-6529646955ec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:30.910336 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.910303 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmmd7\" (UniqueName: \"kubernetes.io/projected/df56e700-da50-4f28-8718-6529646955ec-kube-api-access-fmmd7\") pod \"monitoring-plugin-5f68df7f58-v8jhw\" (UID: \"df56e700-da50-4f28-8718-6529646955ec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:30.910794 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.910354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df56e700-da50-4f28-8718-6529646955ec-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-v8jhw\" (UID: \"df56e700-da50-4f28-8718-6529646955ec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:30.913081 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.913052 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/df56e700-da50-4f28-8718-6529646955ec-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-v8jhw\" (UID: \"df56e700-da50-4f28-8718-6529646955ec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:30.921411 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:30.921386 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmmd7\" (UniqueName: \"kubernetes.io/projected/df56e700-da50-4f28-8718-6529646955ec-kube-api-access-fmmd7\") pod \"monitoring-plugin-5f68df7f58-v8jhw\" (UID: \"df56e700-da50-4f28-8718-6529646955ec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:31.008498 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.008399 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:31.242561 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.242495 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw"] Jul 14 17:06:31.266807 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.266786 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-746b495b48-kfwnf"] Jul 14 17:06:31.276338 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.276307 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.279271 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.279238 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wmzt2\"" Jul 14 17:06:31.279379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.279298 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 17:06:31.279379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.279319 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 17:06:31.279379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.279357 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 17:06:31.279379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.279377 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 17:06:31.279573 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.279318 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 17:06:31.285407 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.285147 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-746b495b48-kfwnf"] Jul 14 17:06:31.286091 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.285975 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 17:06:31.413889 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.413854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.414038 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.413901 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-metrics-client-ca\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.414038 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.413926 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-federate-client-tls\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.414038 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.413949 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmglw\" (UniqueName: \"kubernetes.io/projected/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-kube-api-access-xmglw\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.414038 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.413978 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-telemeter-client-tls\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.414038 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.414001 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-secret-telemeter-client\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.414200 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.414068 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.414200 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.414129 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-serving-certs-ca-bundle\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515134 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-metrics-client-ca\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515134 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-federate-client-tls\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515134 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515106 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmglw\" (UniqueName: \"kubernetes.io/projected/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-kube-api-access-xmglw\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515134 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-telemeter-client-tls\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515445 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515154 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-secret-telemeter-client\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515445 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515182 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515445 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515238 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-serving-certs-ca-bundle\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.515445 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.515321 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.516115 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.516084 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-metrics-client-ca\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.516235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.516141 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-serving-certs-ca-bundle\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.516235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.516149 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.517840 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.517810 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-secret-telemeter-client\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.517971 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.517899 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.518012 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.517989 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-telemeter-client-tls\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.518078 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.518062 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-federate-client-tls\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.530514 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.530487 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmglw\" (UniqueName: \"kubernetes.io/projected/ab2a64e7-c20b-4fcd-9bd4-4775679845fc-kube-api-access-xmglw\") pod \"telemeter-client-746b495b48-kfwnf\" (UID: \"ab2a64e7-c20b-4fcd-9bd4-4775679845fc\") " pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.588214 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.588177 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" Jul 14 17:06:31.762962 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.762934 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-746b495b48-kfwnf"] Jul 14 17:06:31.911471 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.911221 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" event={"ID":"fc9d977a-1414-4de2-97eb-cf9368d51887","Type":"ContainerStarted","Data":"72d40c0ee4b2f023761edbe61c83f0dceb680a3e7d0e6f8a286f444e900c2d5b"} Jul 14 17:06:31.911471 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.911322 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" event={"ID":"fc9d977a-1414-4de2-97eb-cf9368d51887","Type":"ContainerStarted","Data":"025a51f055a751166ff3e605f1517cf4c4dba7664dcc03008749a2fdbb9c86a4"} Jul 14 17:06:31.911471 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.911365 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" event={"ID":"fc9d977a-1414-4de2-97eb-cf9368d51887","Type":"ContainerStarted","Data":"e95f5b95d0ae7c494700d535a02c598320fd2d82917c685b5640587a217c923e"} Jul 14 17:06:31.912880 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:31.912847 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" event={"ID":"df56e700-da50-4f28-8718-6529646955ec","Type":"ContainerStarted","Data":"2d25736236ae5c6b375b367d2b2ce9b986c94a4413d7a7aeeab29b9aa494084d"} Jul 14 17:06:32.051074 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:32.051035 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2a64e7_c20b_4fcd_9bd4_4775679845fc.slice/crio-85efc13b66af502e8b602bc771371ba10b6e89d5350484b4f2fc957f380464d5 WatchSource:0}: Error finding container 85efc13b66af502e8b602bc771371ba10b6e89d5350484b4f2fc957f380464d5: Status 404 returned error can't find the container with id 85efc13b66af502e8b602bc771371ba10b6e89d5350484b4f2fc957f380464d5 Jul 14 17:06:32.270272 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.270244 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:06:32.274217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.274195 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.276816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.276771 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 17:06:32.277066 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.277023 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 17:06:32.277744 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.277710 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 17:06:32.277744 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.277724 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 17:06:32.278006 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.277725 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 17:06:32.278247 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278226 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 17:06:32.278464 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278446 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-64dctuuslokoi\"" Jul 14 17:06:32.278584 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278525 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 17:06:32.278822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278679 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 17:06:32.278822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278450 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 17:06:32.278822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278748 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 17:06:32.278822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278775 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 17:06:32.279100 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.278967 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 17:06:32.279100 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.279040 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-znklr\"" Jul 14 17:06:32.279100 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.279089 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 17:06:32.279250 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.279048 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 17:06:32.285645 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.285616 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 17:06:32.292512 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.292385 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:06:32.322668 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.322668 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322671 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.322668 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322696 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.322668 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322714 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322729 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322825 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322862 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322883 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-web-config\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322911 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322975 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.322990 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gv7m\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-kube-api-access-6gv7m\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323014 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-config\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323039 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323106 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323132 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-config-out\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323342 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323161 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323996 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323191 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.323996 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.323208 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424042 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.423951 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424042 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.423995 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424042 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424020 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-web-config\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424048 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424075 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424126 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424142 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gv7m\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-kube-api-access-6gv7m\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424159 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-config\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424186 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424220 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424252 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424285 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424278 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-config-out\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424326 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424368 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424426 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424452 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424489 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424521 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.424656 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424561 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.425038 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.424871 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.425451 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.425141 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.425451 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.425417 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.426733 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.426706 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.427385 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.427362 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.428082 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.428060 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.428231 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.428208 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.428334 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.428310 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.429030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.429009 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.429816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.429745 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.430510 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.429978 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-config\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.430510 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.430217 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-config-out\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.430510 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.430366 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.430510 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.430418 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.430510 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.430473 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.430824 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.430727 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.431050 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.431019 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.431213 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.431194 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-web-config\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.431908 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.431886 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.434885 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.434866 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gv7m\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-kube-api-access-6gv7m\") pod \"prometheus-k8s-0\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.597744 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.597706 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:32.829839 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.829794 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:06:32.835077 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:06:32.835042 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode013d54d_30c0_4c25_a444_82abb7399277.slice/crio-b2b6b700e27c5d5bb07c567a1cd06898fb76583c3e4d81a46da96202a7db8c9e WatchSource:0}: Error finding container b2b6b700e27c5d5bb07c567a1cd06898fb76583c3e4d81a46da96202a7db8c9e: Status 404 returned error can't find the container with id b2b6b700e27c5d5bb07c567a1cd06898fb76583c3e4d81a46da96202a7db8c9e Jul 14 17:06:32.926415 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.926316 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" event={"ID":"ab2a64e7-c20b-4fcd-9bd4-4775679845fc","Type":"ContainerStarted","Data":"85efc13b66af502e8b602bc771371ba10b6e89d5350484b4f2fc957f380464d5"} Jul 14 17:06:32.929814 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.929576 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} Jul 14 17:06:32.929814 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.929659 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"b2b6b700e27c5d5bb07c567a1cd06898fb76583c3e4d81a46da96202a7db8c9e"} Jul 14 17:06:32.938225 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.938134 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerStarted","Data":"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f"} Jul 14 17:06:32.938379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.938232 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerStarted","Data":"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45"} Jul 14 17:06:32.938379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.938281 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerStarted","Data":"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4"} Jul 14 17:06:32.938379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.938326 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerStarted","Data":"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185"} Jul 14 17:06:32.938379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.938365 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerStarted","Data":"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299"} Jul 14 17:06:32.941263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.941158 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" event={"ID":"df56e700-da50-4f28-8718-6529646955ec","Type":"ContainerStarted","Data":"515fc0dd7074ce15c902ba14afaa11deb2fd28355422ed828c2874e40a2f2dbe"} Jul 14 17:06:32.942083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.941885 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:32.949588 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.949429 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" Jul 14 17:06:32.979116 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:32.979048 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-v8jhw" podStartSLOduration=1.547489151 podStartE2EDuration="2.979024817s" totalImagesPullingTime="1.431535666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:32.978030695 +0000 UTC m=+95.547441468" watchObservedRunningTime="2026-07-14 17:06:32.979024817 +0000 UTC m=+95.548435589" Jul 14 17:06:33.948989 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:33.948947 2558 generic.go:356] "Generic (PLEG): container finished" podID="e013d54d-30c0-4c25-a444-82abb7399277" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" exitCode=0 Jul 14 17:06:33.949430 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:33.949036 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} Jul 14 17:06:33.953586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:33.953526 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerStarted","Data":"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a"} Jul 14 17:06:33.957304 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:33.957275 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" event={"ID":"fc9d977a-1414-4de2-97eb-cf9368d51887","Type":"ContainerStarted","Data":"3db472c3c828cc531120ec7c913f451a64d403e5a887e11ab96045cfce5dad67"} Jul 14 17:06:33.957426 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:33.957356 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" event={"ID":"fc9d977a-1414-4de2-97eb-cf9368d51887","Type":"ContainerStarted","Data":"0637ed6a5f18a93bd6b9fc801908853d67e65476d55a79e3120958e675db399c"} Jul 14 17:06:33.957426 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:33.957403 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" event={"ID":"fc9d977a-1414-4de2-97eb-cf9368d51887","Type":"ContainerStarted","Data":"e2f6b3a870b86dc432d73a25a43e889f5b2deef2c0ddff59ee36c318998fe631"} Jul 14 17:06:34.008510 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:34.008441 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" podStartSLOduration=1.599419371 podStartE2EDuration="6.008422709s" totalImagesPullingTime="4.409003338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:34.006491501 +0000 UTC m=+96.575902274" watchObservedRunningTime="2026-07-14 17:06:34.008422709 +0000 UTC m=+96.577833481" Jul 14 17:06:34.039925 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:34.039859 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.005649891 podStartE2EDuration="7.03984256s" totalImagesPullingTime="4.014566869s" totalInitContainerRuntime="19.6258ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:34.037523895 +0000 UTC m=+96.606934666" watchObservedRunningTime="2026-07-14 17:06:34.03984256 +0000 UTC m=+96.609253331" Jul 14 17:06:34.966596 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:34.966470 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" event={"ID":"ab2a64e7-c20b-4fcd-9bd4-4775679845fc","Type":"ContainerStarted","Data":"c763ac025efa1db4e9746e402e4134bce490a798ac08e6f7d5210bfd3d592632"} Jul 14 17:06:34.966596 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:34.966558 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" event={"ID":"ab2a64e7-c20b-4fcd-9bd4-4775679845fc","Type":"ContainerStarted","Data":"63618f070940f9c804abe34e1f740d3a4c8c892aea77ce6d5025661af4addc92"} Jul 14 17:06:34.966596 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:34.966592 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" event={"ID":"ab2a64e7-c20b-4fcd-9bd4-4775679845fc","Type":"ContainerStarted","Data":"aaf6e956ef91927a47371f70ad0bd9e1240688860b9145c2cd60485fbee1270f"} Jul 14 17:06:34.968179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:34.968148 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:34.992103 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:34.992045 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-746b495b48-kfwnf" podStartSLOduration=1.471077572 podStartE2EDuration="3.992025976s" totalImagesPullingTime="2.520948404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:34.989889094 +0000 UTC m=+97.559299899" watchObservedRunningTime="2026-07-14 17:06:34.992025976 +0000 UTC m=+97.561436746" Jul 14 17:06:36.742719 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:36.742683 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-47glh" Jul 14 17:06:38.991840 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:38.991800 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} Jul 14 17:06:38.992220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:38.991908 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} Jul 14 17:06:38.992220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:38.991944 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} Jul 14 17:06:38.992220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:38.991969 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} Jul 14 17:06:38.992220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:38.992000 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} Jul 14 17:06:38.992220 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:38.992032 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerStarted","Data":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} Jul 14 17:06:39.028806 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:39.028750 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.708827204 podStartE2EDuration="7.028732818s" totalImagesPullingTime="4.289653027s" totalInitContainerRuntime="30.252587ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:39.025686675 +0000 UTC m=+101.595097483" watchObservedRunningTime="2026-07-14 17:06:39.028732818 +0000 UTC m=+101.598143589" Jul 14 17:06:40.979089 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:40.979058 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5bf6fdfc65-rr5pc" Jul 14 17:06:42.598720 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:42.598686 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:06:55.068126 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.068089 2558 generic.go:356] "Generic (PLEG): container finished" podID="df83cbc3-0758-4506-8e21-6641839d5f8b" containerID="234acac67c5d0ed9e9c72b719c425f2dfdee6842d37ea550bcca9d16f0cc4674" exitCode=0 Jul 14 17:06:55.068715 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.068151 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" event={"ID":"df83cbc3-0758-4506-8e21-6641839d5f8b","Type":"ContainerDied","Data":"234acac67c5d0ed9e9c72b719c425f2dfdee6842d37ea550bcca9d16f0cc4674"} Jul 14 17:06:55.068715 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.068663 2558 scope.go:117] "RemoveContainer" containerID="234acac67c5d0ed9e9c72b719c425f2dfdee6842d37ea550bcca9d16f0cc4674" Jul 14 17:06:55.158005 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.157933 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" podUID="bc87436a-95c3-40e5-9470-6acaf43aabbe" containerName="registry" containerID="cri-o://bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f" gracePeriod=30 Jul 14 17:06:55.418750 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.418722 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:06:55.456076 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456048 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc87436a-95c3-40e5-9470-6acaf43aabbe-ca-trust-extracted\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.456239 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456117 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-image-registry-private-configuration\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.456239 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456152 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.456239 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456205 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-installation-pull-secrets\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.456389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456258 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-certificates\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.456389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456303 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2t6c\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-kube-api-access-k2t6c\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.456389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456327 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-bound-sa-token\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.456389 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.456350 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-trusted-ca\") pod \"bc87436a-95c3-40e5-9470-6acaf43aabbe\" (UID: \"bc87436a-95c3-40e5-9470-6acaf43aabbe\") " Jul 14 17:06:55.457068 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.457040 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:06:55.457208 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.457058 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:06:55.459399 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.459362 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:06:55.459510 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.459399 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:06:55.459640 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.459622 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-kube-api-access-k2t6c" (OuterVolumeSpecName: "kube-api-access-k2t6c") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "kube-api-access-k2t6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:06:55.459765 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.459747 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:06:55.459947 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.459925 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:06:55.465672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.465642 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc87436a-95c3-40e5-9470-6acaf43aabbe-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bc87436a-95c3-40e5-9470-6acaf43aabbe" (UID: "bc87436a-95c3-40e5-9470-6acaf43aabbe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:06:55.557327 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557265 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-image-registry-private-configuration\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:55.557327 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557321 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:55.557589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557351 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bc87436a-95c3-40e5-9470-6acaf43aabbe-installation-pull-secrets\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:55.557589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557365 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-registry-certificates\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:55.557589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557379 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2t6c\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-kube-api-access-k2t6c\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:55.557589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557392 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc87436a-95c3-40e5-9470-6acaf43aabbe-bound-sa-token\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:55.557589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557405 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc87436a-95c3-40e5-9470-6acaf43aabbe-trusted-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:55.557589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:55.557417 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bc87436a-95c3-40e5-9470-6acaf43aabbe-ca-trust-extracted\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:06:56.073946 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.073911 2558 generic.go:356] "Generic (PLEG): container finished" podID="bc87436a-95c3-40e5-9470-6acaf43aabbe" containerID="bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f" exitCode=0 Jul 14 17:06:56.074429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.073957 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" Jul 14 17:06:56.074429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.074004 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" event={"ID":"bc87436a-95c3-40e5-9470-6acaf43aabbe","Type":"ContainerDied","Data":"bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f"} Jul 14 17:06:56.074429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.074057 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb86598f-m8z72" event={"ID":"bc87436a-95c3-40e5-9470-6acaf43aabbe","Type":"ContainerDied","Data":"0332bcf31085a42afecc5f17f610f38bdd78aa38dbca02637c3ccf2dbb838e47"} Jul 14 17:06:56.074429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.074073 2558 scope.go:117] "RemoveContainer" containerID="bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f" Jul 14 17:06:56.076152 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.076122 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zlgcw" event={"ID":"df83cbc3-0758-4506-8e21-6641839d5f8b","Type":"ContainerStarted","Data":"293986c16fdbdb64171cbcb7868e904dc722f9d8f78a0daef8403ce9dddb8d54"} Jul 14 17:06:56.084801 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.084711 2558 scope.go:117] "RemoveContainer" containerID="bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f" Jul 14 17:06:56.085228 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:06:56.085141 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f\": container with ID starting with bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f not found: ID does not exist" containerID="bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f" Jul 14 17:06:56.085228 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.085179 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f"} err="failed to get container status \"bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f\": rpc error: code = NotFound desc = could not find container \"bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f\": container with ID starting with bbf392273feafcd329b27843296d2c8dba4c2d4e18ea61511988f247892ef90f not found: ID does not exist" Jul 14 17:06:56.125740 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.125702 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fb86598f-m8z72"] Jul 14 17:06:56.131271 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:56.131247 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6fb86598f-m8z72"] Jul 14 17:06:57.387226 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:06:57.387189 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc87436a-95c3-40e5-9470-6acaf43aabbe" path="/var/lib/kubelet/pods/bc87436a-95c3-40e5-9470-6acaf43aabbe/volumes" Jul 14 17:07:15.172303 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:15.172264 2558 generic.go:356] "Generic (PLEG): container finished" podID="75d34625-d739-4296-be29-f90c173c1a9c" containerID="601e3adca0dd310214e71ea235670768ca8318f6ad73eee5a9b74b958d9f967b" exitCode=0 Jul 14 17:07:15.172860 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:15.172329 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" event={"ID":"75d34625-d739-4296-be29-f90c173c1a9c","Type":"ContainerDied","Data":"601e3adca0dd310214e71ea235670768ca8318f6ad73eee5a9b74b958d9f967b"} Jul 14 17:07:15.172860 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:15.172777 2558 scope.go:117] "RemoveContainer" containerID="601e3adca0dd310214e71ea235670768ca8318f6ad73eee5a9b74b958d9f967b" Jul 14 17:07:16.178965 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:16.178931 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-px8zl" event={"ID":"75d34625-d739-4296-be29-f90c173c1a9c","Type":"ContainerStarted","Data":"bc54fbe632fc3ee625a8ea5a9c7af8ac2045c1bb17a917be9d48956ccb8c2bcd"} Jul 14 17:07:20.200167 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:20.200133 2558 generic.go:356] "Generic (PLEG): container finished" podID="f42ac6be-eba5-4f04-99d6-be1b2e3c9e98" containerID="2e0814d50f14c70547bd1c4b20e2f92cdcdb759202977fe7fe500b4ca7629abf" exitCode=0 Jul 14 17:07:20.200674 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:20.200216 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" event={"ID":"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98","Type":"ContainerDied","Data":"2e0814d50f14c70547bd1c4b20e2f92cdcdb759202977fe7fe500b4ca7629abf"} Jul 14 17:07:20.200674 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:20.200589 2558 scope.go:117] "RemoveContainer" containerID="2e0814d50f14c70547bd1c4b20e2f92cdcdb759202977fe7fe500b4ca7629abf" Jul 14 17:07:21.211955 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:21.211919 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-wgq9k" event={"ID":"f42ac6be-eba5-4f04-99d6-be1b2e3c9e98","Type":"ContainerStarted","Data":"59401f7ae09f43b2b0bd8c475a2c6567b29cccca9d9d74073e5fdf7e39a0e45f"} Jul 14 17:07:32.598099 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:32.598030 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:32.616561 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:32.616518 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:33.280288 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:33.280256 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:46.469161 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:46.469125 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:07:46.469657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:46.469408 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="alertmanager" containerID="cri-o://48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299" gracePeriod=120 Jul 14 17:07:46.469657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:46.469471 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-web" containerID="cri-o://33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4" gracePeriod=120 Jul 14 17:07:46.469657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:46.469490 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-metric" containerID="cri-o://27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f" gracePeriod=120 Jul 14 17:07:46.469657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:46.469574 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="config-reloader" containerID="cri-o://69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185" gracePeriod=120 Jul 14 17:07:46.469657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:46.469608 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="prom-label-proxy" containerID="cri-o://8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a" gracePeriod=120 Jul 14 17:07:46.469657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:46.469508 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy" containerID="cri-o://74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45" gracePeriod=120 Jul 14 17:07:47.328599 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328560 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7903c5-3b96-4915-929e-377f4e8da338" containerID="8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a" exitCode=0 Jul 14 17:07:47.328599 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328600 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7903c5-3b96-4915-929e-377f4e8da338" containerID="74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45" exitCode=0 Jul 14 17:07:47.328833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328611 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7903c5-3b96-4915-929e-377f4e8da338" containerID="69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185" exitCode=0 Jul 14 17:07:47.328833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328620 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7903c5-3b96-4915-929e-377f4e8da338" containerID="48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299" exitCode=0 Jul 14 17:07:47.328833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328573 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a"} Jul 14 17:07:47.328833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328726 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45"} Jul 14 17:07:47.328833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328756 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185"} Jul 14 17:07:47.328833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.328780 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299"} Jul 14 17:07:47.728522 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.728497 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:47.749688 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749656 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-web\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.749864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749701 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-cluster-tls-config\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.749864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749733 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-tls-assets\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.749864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749764 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-main-tls\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.749864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749798 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-metrics-client-ca\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.749864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749821 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-web-config\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.749864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749848 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-trusted-ca-bundle\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.750171 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749899 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.750171 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749927 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-main-db\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.750171 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.749992 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-config-volume\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.750171 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.750024 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-config-out\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.750171 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.750089 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4jf9\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-kube-api-access-d4jf9\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.750171 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.750134 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy\") pod \"ed7903c5-3b96-4915-929e-377f4e8da338\" (UID: \"ed7903c5-3b96-4915-929e-377f4e8da338\") " Jul 14 17:07:47.750452 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.750267 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:47.750505 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.750463 2558 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-metrics-client-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.751168 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.750673 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:47.751168 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.751124 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:07:47.753504 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.753048 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:47.753504 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.753350 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-config-volume" (OuterVolumeSpecName: "config-volume") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:47.753504 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.753382 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-config-out" (OuterVolumeSpecName: "config-out") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:07:47.753504 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.753461 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:07:47.753821 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.753794 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:47.754178 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.754136 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-kube-api-access-d4jf9" (OuterVolumeSpecName: "kube-api-access-d4jf9") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "kube-api-access-d4jf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:07:47.754416 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.754393 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:47.754948 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.754925 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:47.759599 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.759571 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:47.778826 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.778790 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-web-config" (OuterVolumeSpecName: "web-config") pod "ed7903c5-3b96-4915-929e-377f4e8da338" (UID: "ed7903c5-3b96-4915-929e-377f4e8da338"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:47.851950 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851872 2558 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-config-volume\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.851950 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851898 2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-config-out\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.851950 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851911 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4jf9\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-kube-api-access-d4jf9\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.851950 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851927 2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.851950 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851942 2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.852217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851955 2558 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-cluster-tls-config\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.852217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851970 2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed7903c5-3b96-4915-929e-377f4e8da338-tls-assets\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.852217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.851982 2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-main-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.852217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.852009 2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-web-config\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.852217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.852018 2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.852217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.852026 2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed7903c5-3b96-4915-929e-377f4e8da338-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:47.852217 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:47.852036 2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed7903c5-3b96-4915-929e-377f4e8da338-alertmanager-main-db\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:48.337280 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.337240 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7903c5-3b96-4915-929e-377f4e8da338" containerID="27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f" exitCode=0 Jul 14 17:07:48.337280 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.337271 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7903c5-3b96-4915-929e-377f4e8da338" containerID="33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4" exitCode=0 Jul 14 17:07:48.337491 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.337297 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f"} Jul 14 17:07:48.337491 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.337343 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.337491 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.337401 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4"} Jul 14 17:07:48.337491 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.337430 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed7903c5-3b96-4915-929e-377f4e8da338","Type":"ContainerDied","Data":"37bc3408de54c9f9552f22b795d2629e427f2ab976e751c8f14ce37f3cead8f2"} Jul 14 17:07:48.337491 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.337460 2558 scope.go:117] "RemoveContainer" containerID="8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a" Jul 14 17:07:48.346536 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.346517 2558 scope.go:117] "RemoveContainer" containerID="27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f" Jul 14 17:07:48.357723 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.357700 2558 scope.go:117] "RemoveContainer" containerID="74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45" Jul 14 17:07:48.363803 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.363773 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:07:48.367841 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.367818 2558 scope.go:117] "RemoveContainer" containerID="33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4" Jul 14 17:07:48.368405 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.368383 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:07:48.375906 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.375884 2558 scope.go:117] "RemoveContainer" containerID="69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185" Jul 14 17:07:48.384174 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.384153 2558 scope.go:117] "RemoveContainer" containerID="48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299" Jul 14 17:07:48.392112 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.392091 2558 scope.go:117] "RemoveContainer" containerID="e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5" Jul 14 17:07:48.400006 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.399891 2558 scope.go:117] "RemoveContainer" containerID="8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a" Jul 14 17:07:48.400300 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:48.400262 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a\": container with ID starting with 8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a not found: ID does not exist" containerID="8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a" Jul 14 17:07:48.400398 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.400305 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a"} err="failed to get container status \"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a\": rpc error: code = NotFound desc = could not find container \"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a\": container with ID starting with 8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a not found: ID does not exist" Jul 14 17:07:48.400398 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.400330 2558 scope.go:117] "RemoveContainer" containerID="27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f" Jul 14 17:07:48.400781 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:48.400762 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f\": container with ID starting with 27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f not found: ID does not exist" containerID="27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f" Jul 14 17:07:48.400866 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.400787 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f"} err="failed to get container status \"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f\": rpc error: code = NotFound desc = could not find container \"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f\": container with ID starting with 27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f not found: ID does not exist" Jul 14 17:07:48.400866 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.400805 2558 scope.go:117] "RemoveContainer" containerID="74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45" Jul 14 17:07:48.401075 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:48.401056 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45\": container with ID starting with 74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45 not found: ID does not exist" containerID="74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45" Jul 14 17:07:48.401115 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401085 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45"} err="failed to get container status \"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45\": rpc error: code = NotFound desc = could not find container \"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45\": container with ID starting with 74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45 not found: ID does not exist" Jul 14 17:07:48.401115 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401100 2558 scope.go:117] "RemoveContainer" containerID="33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4" Jul 14 17:07:48.401336 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:48.401321 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4\": container with ID starting with 33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4 not found: ID does not exist" containerID="33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4" Jul 14 17:07:48.401384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401339 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4"} err="failed to get container status \"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4\": rpc error: code = NotFound desc = could not find container \"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4\": container with ID starting with 33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4 not found: ID does not exist" Jul 14 17:07:48.401384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401350 2558 scope.go:117] "RemoveContainer" containerID="69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185" Jul 14 17:07:48.401612 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:48.401596 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185\": container with ID starting with 69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185 not found: ID does not exist" containerID="69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185" Jul 14 17:07:48.401665 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401615 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185"} err="failed to get container status \"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185\": rpc error: code = NotFound desc = could not find container \"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185\": container with ID starting with 69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185 not found: ID does not exist" Jul 14 17:07:48.401665 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401627 2558 scope.go:117] "RemoveContainer" containerID="48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299" Jul 14 17:07:48.401844 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:48.401828 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299\": container with ID starting with 48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299 not found: ID does not exist" containerID="48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299" Jul 14 17:07:48.401883 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401847 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299"} err="failed to get container status \"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299\": rpc error: code = NotFound desc = could not find container \"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299\": container with ID starting with 48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299 not found: ID does not exist" Jul 14 17:07:48.401883 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.401861 2558 scope.go:117] "RemoveContainer" containerID="e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5" Jul 14 17:07:48.402082 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:48.402063 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5\": container with ID starting with e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5 not found: ID does not exist" containerID="e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5" Jul 14 17:07:48.402161 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402088 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5"} err="failed to get container status \"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5\": rpc error: code = NotFound desc = could not find container \"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5\": container with ID starting with e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5 not found: ID does not exist" Jul 14 17:07:48.402161 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402103 2558 scope.go:117] "RemoveContainer" containerID="8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a" Jul 14 17:07:48.402308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402293 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a"} err="failed to get container status \"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a\": rpc error: code = NotFound desc = could not find container \"8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a\": container with ID starting with 8f449d7b38e8c779cb8e2a2310d3d24fa5bfd79f23d43aa213f39f34fa40889a not found: ID does not exist" Jul 14 17:07:48.402345 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402309 2558 scope.go:117] "RemoveContainer" containerID="27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f" Jul 14 17:07:48.402487 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402470 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f"} err="failed to get container status \"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f\": rpc error: code = NotFound desc = could not find container \"27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f\": container with ID starting with 27dd72000d14f536153939397593c52247fb2710d7dea0a8a48ceff11079f76f not found: ID does not exist" Jul 14 17:07:48.402524 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402488 2558 scope.go:117] "RemoveContainer" containerID="74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45" Jul 14 17:07:48.402700 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402682 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45"} err="failed to get container status \"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45\": rpc error: code = NotFound desc = could not find container \"74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45\": container with ID starting with 74358c5613bd087609b12bde0d458929bfc6594a18c91f8c6bacce8cf1253b45 not found: ID does not exist" Jul 14 17:07:48.402754 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402700 2558 scope.go:117] "RemoveContainer" containerID="33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4" Jul 14 17:07:48.402932 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402911 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4"} err="failed to get container status \"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4\": rpc error: code = NotFound desc = could not find container \"33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4\": container with ID starting with 33ca285cbc7c0a4c2ae31cce31e2b3401869f26c9babc02a69c906639a0729b4 not found: ID does not exist" Jul 14 17:07:48.403002 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.402934 2558 scope.go:117] "RemoveContainer" containerID="69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185" Jul 14 17:07:48.403147 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.403127 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185"} err="failed to get container status \"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185\": rpc error: code = NotFound desc = could not find container \"69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185\": container with ID starting with 69421b5f6d6f8d908a60636752c636267c92560fbdfc1407f9072c937c2eb185 not found: ID does not exist" Jul 14 17:07:48.403195 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.403148 2558 scope.go:117] "RemoveContainer" containerID="48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299" Jul 14 17:07:48.403349 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.403331 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299"} err="failed to get container status \"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299\": rpc error: code = NotFound desc = could not find container \"48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299\": container with ID starting with 48e0ee76ece6cf3aaebc8be8d2c2ef485aaeb76ef11f2e4f679dc8725372a299 not found: ID does not exist" Jul 14 17:07:48.403420 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.403351 2558 scope.go:117] "RemoveContainer" containerID="e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5" Jul 14 17:07:48.403583 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.403565 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5"} err="failed to get container status \"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5\": rpc error: code = NotFound desc = could not find container \"e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5\": container with ID starting with e9b728b6463563ad902715a400f32ceda6aec255db08bede46c7576f4dbb12b5 not found: ID does not exist" Jul 14 17:07:48.413266 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413241 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:07:48.413668 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413653 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="alertmanager" Jul 14 17:07:48.413756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413670 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="alertmanager" Jul 14 17:07:48.413756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413685 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="prom-label-proxy" Jul 14 17:07:48.413756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413693 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="prom-label-proxy" Jul 14 17:07:48.413756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413706 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-web" Jul 14 17:07:48.413756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413715 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-web" Jul 14 17:07:48.413756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413733 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy" Jul 14 17:07:48.413756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413742 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413759 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="config-reloader" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413767 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="config-reloader" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413782 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc87436a-95c3-40e5-9470-6acaf43aabbe" containerName="registry" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413791 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc87436a-95c3-40e5-9470-6acaf43aabbe" containerName="registry" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413805 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="init-config-reloader" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413814 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="init-config-reloader" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413840 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-metric" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413849 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-metric" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413944 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413957 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-metric" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413969 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc87436a-95c3-40e5-9470-6acaf43aabbe" containerName="registry" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413980 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="config-reloader" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413989 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="kube-rbac-proxy-web" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.413999 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="prom-label-proxy" Jul 14 17:07:48.414079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.414008 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" containerName="alertmanager" Jul 14 17:07:48.419752 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.419733 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.423676 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.423655 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 17:07:48.423676 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.423655 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 17:07:48.423919 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.423901 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 17:07:48.423985 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.423951 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 17:07:48.424047 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.423987 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rllwh\"" Jul 14 17:07:48.424047 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.424035 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 17:07:48.424353 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.424337 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 17:07:48.428909 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.428890 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 17:07:48.435631 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.435614 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 17:07:48.436161 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.436147 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457220 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afc7dab-5f74-4b95-9690-a9af36743454-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457265 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3afc7dab-5f74-4b95-9690-a9af36743454-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457304 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457352 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457382 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-config-volume\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457411 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457441 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457473 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3afc7dab-5f74-4b95-9690-a9af36743454-config-out\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.457586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-web-config\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.458147 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457659 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3afc7dab-5f74-4b95-9690-a9af36743454-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.458147 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457706 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3afc7dab-5f74-4b95-9690-a9af36743454-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.458147 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457808 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.458147 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.457831 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rph\" (UniqueName: \"kubernetes.io/projected/3afc7dab-5f74-4b95-9690-a9af36743454-kube-api-access-h7rph\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.459283 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.459263 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:07:48.559328 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559297 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559492 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559349 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559562 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-config-volume\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559610 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559610 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559600 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559705 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3afc7dab-5f74-4b95-9690-a9af36743454-config-out\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559705 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-web-config\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559803 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559706 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3afc7dab-5f74-4b95-9690-a9af36743454-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559803 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559747 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3afc7dab-5f74-4b95-9690-a9af36743454-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559803 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559793 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559955 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559815 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rph\" (UniqueName: \"kubernetes.io/projected/3afc7dab-5f74-4b95-9690-a9af36743454-kube-api-access-h7rph\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559955 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559875 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afc7dab-5f74-4b95-9690-a9af36743454-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.559955 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.559903 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3afc7dab-5f74-4b95-9690-a9af36743454-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.560638 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.560599 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3afc7dab-5f74-4b95-9690-a9af36743454-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.560763 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.560716 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3afc7dab-5f74-4b95-9690-a9af36743454-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.561432 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.561406 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afc7dab-5f74-4b95-9690-a9af36743454-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.562474 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.562448 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-config-volume\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.562608 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.562522 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.562881 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.562856 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.562999 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.562948 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.563194 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.563171 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3afc7dab-5f74-4b95-9690-a9af36743454-config-out\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.563326 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.563306 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-web-config\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.563387 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.563368 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.563747 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.563729 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3afc7dab-5f74-4b95-9690-a9af36743454-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.564414 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.564393 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3afc7dab-5f74-4b95-9690-a9af36743454-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.570078 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.570052 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rph\" (UniqueName: \"kubernetes.io/projected/3afc7dab-5f74-4b95-9690-a9af36743454-kube-api-access-h7rph\") pod \"alertmanager-main-0\" (UID: \"3afc7dab-5f74-4b95-9690-a9af36743454\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.729780 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.729681 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:07:48.874535 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:48.874506 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:07:48.876366 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:07:48.876342 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3afc7dab_5f74_4b95_9690_a9af36743454.slice/crio-4952b82ae9a134563c6ce8fae082e4d4923c6eb3e2aee3ae0dab2973dfa673bc WatchSource:0}: Error finding container 4952b82ae9a134563c6ce8fae082e4d4923c6eb3e2aee3ae0dab2973dfa673bc: Status 404 returned error can't find the container with id 4952b82ae9a134563c6ce8fae082e4d4923c6eb3e2aee3ae0dab2973dfa673bc Jul 14 17:07:49.343479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:49.343402 2558 generic.go:356] "Generic (PLEG): container finished" podID="3afc7dab-5f74-4b95-9690-a9af36743454" containerID="e090f98c8fbc5117ffcf7880f911e3c914695ded6654cdb40a9d96f6435ef412" exitCode=0 Jul 14 17:07:49.343479 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:49.343440 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerDied","Data":"e090f98c8fbc5117ffcf7880f911e3c914695ded6654cdb40a9d96f6435ef412"} Jul 14 17:07:49.343697 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:49.343491 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerStarted","Data":"4952b82ae9a134563c6ce8fae082e4d4923c6eb3e2aee3ae0dab2973dfa673bc"} Jul 14 17:07:49.387831 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:49.387801 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed7903c5-3b96-4915-929e-377f4e8da338" path="/var/lib/kubelet/pods/ed7903c5-3b96-4915-929e-377f4e8da338/volumes" Jul 14 17:07:50.353154 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.353119 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerStarted","Data":"5dcb003ad84d69e9c13b1af01208c53a79d55f67bce66f971eb0e2d86cc0beb6"} Jul 14 17:07:50.353556 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.353185 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerStarted","Data":"1d0329a6ddd104431d549ad670db8c600a69bb64769343ddeef4e8eda6209fad"} Jul 14 17:07:50.353556 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.353217 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerStarted","Data":"1503226f0df2c52b39ddea9226e5062b27630a69193ecbaf2f90a4d2421fd0b5"} Jul 14 17:07:50.353556 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.353242 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerStarted","Data":"cb94c759f2e3c9baf9cb664bd4cd4995ad89733250ea1413c8da0bda10ae7355"} Jul 14 17:07:50.353556 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.353264 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerStarted","Data":"dfad4f48843601b2ea8109ba809225f817493df3aa562de1aa791b4355e3bea3"} Jul 14 17:07:50.353556 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.353287 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3afc7dab-5f74-4b95-9690-a9af36743454","Type":"ContainerStarted","Data":"577ba5eb796b3828cd4edf9da5e4dc8ea8ba31b7223fdeb948428a8a9e08f3f1"} Jul 14 17:07:50.382122 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.382061 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.367392585 podStartE2EDuration="2.382045391s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.652806ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:07:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:07:50.381033101 +0000 UTC m=+172.950443893" watchObservedRunningTime="2026-07-14 17:07:50.382045391 +0000 UTC m=+172.951456144" Jul 14 17:07:50.721214 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.721129 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:07:50.721487 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.721426 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="prometheus" containerID="cri-o://ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" gracePeriod=600 Jul 14 17:07:50.721487 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.721450 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy" containerID="cri-o://179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" gracePeriod=600 Jul 14 17:07:50.721657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.721487 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-web" containerID="cri-o://c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" gracePeriod=600 Jul 14 17:07:50.721657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.721453 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="thanos-sidecar" containerID="cri-o://416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" gracePeriod=600 Jul 14 17:07:50.721657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.721460 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="config-reloader" containerID="cri-o://086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" gracePeriod=600 Jul 14 17:07:50.721657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.721510 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-thanos" containerID="cri-o://950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" gracePeriod=600 Jul 14 17:07:50.962528 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:50.962501 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.086419 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086376 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086419 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086424 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-serving-certs-ca-bundle\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086454 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-kube-rbac-proxy\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086482 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-trusted-ca-bundle\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086509 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-web-config\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086535 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-config\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086570 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-grpc-tls\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086597 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-0\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086644 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-2\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.086679 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086671 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gv7m\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-kube-api-access-6gv7m\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086695 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-kubelet-serving-ca-bundle\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086761 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-tls\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086790 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-thanos-prometheus-http-client-file\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086821 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-db\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086842 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-config-out\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086864 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-metrics-client-certs\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086893 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-1\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086939 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-metrics-client-ca\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.086993 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-tls-assets\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.087025 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e013d54d-30c0-4c25-a444-82abb7399277\" (UID: \"e013d54d-30c0-4c25-a444-82abb7399277\") " Jul 14 17:07:51.087083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.087042 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:51.087657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.087344 2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.087983 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.087952 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:51.088257 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.088191 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:07:51.088794 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.088500 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:51.088915 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.088798 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:51.089267 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.089238 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.089390 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.089316 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.089390 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.089371 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-config" (OuterVolumeSpecName: "config") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.089663 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.089627 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:51.089776 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.089662 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.089776 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.089723 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:51.091123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.091091 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-kube-api-access-6gv7m" (OuterVolumeSpecName: "kube-api-access-6gv7m") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "kube-api-access-6gv7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:07:51.091222 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.091127 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.091525 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.091477 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:51.091633 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.091596 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-config-out" (OuterVolumeSpecName: "config-out") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:07:51.091774 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.091747 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.091974 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.091955 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.092028 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.092013 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.092099 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.092081 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:07:51.101964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.101932 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-web-config" (OuterVolumeSpecName: "web-config") pod "e013d54d-30c0-4c25-a444-82abb7399277" (UID: "e013d54d-30c0-4c25-a444-82abb7399277"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:51.188184 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188151 2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188184 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188183 2558 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188184 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188193 2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-db\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188203 2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e013d54d-30c0-4c25-a444-82abb7399277-config-out\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188213 2558 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-metrics-client-certs\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188222 2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188232 2558 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-metrics-client-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188240 2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-tls-assets\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188251 2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188260 2558 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188269 2558 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188278 2558 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-kube-rbac-proxy\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188286 2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-web-config\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188294 2558 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-config\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188302 2558 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e013d54d-30c0-4c25-a444-82abb7399277-secret-grpc-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188309 2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188317 2558 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188325 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gv7m\" (UniqueName: \"kubernetes.io/projected/e013d54d-30c0-4c25-a444-82abb7399277-kube-api-access-6gv7m\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.188397 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.188333 2558 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e013d54d-30c0-4c25-a444-82abb7399277-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:07:51.361316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361224 2558 generic.go:356] "Generic (PLEG): container finished" podID="e013d54d-30c0-4c25-a444-82abb7399277" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" exitCode=0 Jul 14 17:07:51.361316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361251 2558 generic.go:356] "Generic (PLEG): container finished" podID="e013d54d-30c0-4c25-a444-82abb7399277" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" exitCode=0 Jul 14 17:07:51.361316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361257 2558 generic.go:356] "Generic (PLEG): container finished" podID="e013d54d-30c0-4c25-a444-82abb7399277" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" exitCode=0 Jul 14 17:07:51.361316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361263 2558 generic.go:356] "Generic (PLEG): container finished" podID="e013d54d-30c0-4c25-a444-82abb7399277" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" exitCode=0 Jul 14 17:07:51.361316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361268 2558 generic.go:356] "Generic (PLEG): container finished" podID="e013d54d-30c0-4c25-a444-82abb7399277" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" exitCode=0 Jul 14 17:07:51.361316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361273 2558 generic.go:356] "Generic (PLEG): container finished" podID="e013d54d-30c0-4c25-a444-82abb7399277" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" exitCode=0 Jul 14 17:07:51.361316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361314 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361312 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361481 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361512 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361559 2558 scope.go:117] "RemoveContainer" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361674 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361705 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} Jul 14 17:07:51.361964 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.361742 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e013d54d-30c0-4c25-a444-82abb7399277","Type":"ContainerDied","Data":"b2b6b700e27c5d5bb07c567a1cd06898fb76583c3e4d81a46da96202a7db8c9e"} Jul 14 17:07:51.371466 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.371445 2558 scope.go:117] "RemoveContainer" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.379356 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.379340 2558 scope.go:117] "RemoveContainer" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.386764 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.386736 2558 scope.go:117] "RemoveContainer" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.390811 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.390783 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:07:51.394440 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.394413 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:07:51.397718 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.397702 2558 scope.go:117] "RemoveContainer" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.405102 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.405086 2558 scope.go:117] "RemoveContainer" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.412913 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.412892 2558 scope.go:117] "RemoveContainer" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.420612 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.420590 2558 scope.go:117] "RemoveContainer" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.420899 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:51.420880 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": container with ID starting with 950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f not found: ID does not exist" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.420965 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.420908 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} err="failed to get container status \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": rpc error: code = NotFound desc = could not find container \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": container with ID starting with 950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f not found: ID does not exist" Jul 14 17:07:51.420965 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.420927 2558 scope.go:117] "RemoveContainer" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.421150 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:51.421130 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": container with ID starting with 179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134 not found: ID does not exist" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.421193 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.421158 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} err="failed to get container status \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": rpc error: code = NotFound desc = could not find container \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": container with ID starting with 179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134 not found: ID does not exist" Jul 14 17:07:51.421193 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.421176 2558 scope.go:117] "RemoveContainer" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.421395 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:51.421380 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": container with ID starting with c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957 not found: ID does not exist" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.421430 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.421398 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} err="failed to get container status \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": rpc error: code = NotFound desc = could not find container \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": container with ID starting with c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957 not found: ID does not exist" Jul 14 17:07:51.421430 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.421411 2558 scope.go:117] "RemoveContainer" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.421665 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:51.421645 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": container with ID starting with 416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45 not found: ID does not exist" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.421665 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.421671 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} err="failed to get container status \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": rpc error: code = NotFound desc = could not find container \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": container with ID starting with 416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45 not found: ID does not exist" Jul 14 17:07:51.421665 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.421691 2558 scope.go:117] "RemoveContainer" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.422611 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:51.422582 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": container with ID starting with 086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1 not found: ID does not exist" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.422689 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.422618 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} err="failed to get container status \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": rpc error: code = NotFound desc = could not find container \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": container with ID starting with 086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1 not found: ID does not exist" Jul 14 17:07:51.422689 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.422639 2558 scope.go:117] "RemoveContainer" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.422931 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:51.422914 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": container with ID starting with ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b not found: ID does not exist" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.422986 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.422935 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} err="failed to get container status \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": rpc error: code = NotFound desc = could not find container \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": container with ID starting with ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b not found: ID does not exist" Jul 14 17:07:51.422986 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.422950 2558 scope.go:117] "RemoveContainer" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.423225 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:07:51.423209 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": container with ID starting with 60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed not found: ID does not exist" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.423301 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423227 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} err="failed to get container status \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": rpc error: code = NotFound desc = could not find container \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": container with ID starting with 60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed not found: ID does not exist" Jul 14 17:07:51.423301 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423241 2558 scope.go:117] "RemoveContainer" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.423496 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423474 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} err="failed to get container status \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": rpc error: code = NotFound desc = could not find container \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": container with ID starting with 950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f not found: ID does not exist" Jul 14 17:07:51.423561 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423499 2558 scope.go:117] "RemoveContainer" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.423756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423734 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} err="failed to get container status \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": rpc error: code = NotFound desc = could not find container \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": container with ID starting with 179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134 not found: ID does not exist" Jul 14 17:07:51.423805 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423758 2558 scope.go:117] "RemoveContainer" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.423990 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423973 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} err="failed to get container status \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": rpc error: code = NotFound desc = could not find container \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": container with ID starting with c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957 not found: ID does not exist" Jul 14 17:07:51.424030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.423992 2558 scope.go:117] "RemoveContainer" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.424219 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424201 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} err="failed to get container status \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": rpc error: code = NotFound desc = could not find container \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": container with ID starting with 416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45 not found: ID does not exist" Jul 14 17:07:51.424269 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424221 2558 scope.go:117] "RemoveContainer" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.424302 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424280 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:07:51.424455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424438 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} err="failed to get container status \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": rpc error: code = NotFound desc = could not find container \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": container with ID starting with 086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1 not found: ID does not exist" Jul 14 17:07:51.424492 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424455 2558 scope.go:117] "RemoveContainer" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.424715 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424698 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424723 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424737 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="thanos-sidecar" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424743 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="thanos-sidecar" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424762 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="config-reloader" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424768 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="config-reloader" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424692 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} err="failed to get container status \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": rpc error: code = NotFound desc = could not find container \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": container with ID starting with ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b not found: ID does not exist" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424782 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="prometheus" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424790 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="prometheus" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424802 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-web" Jul 14 17:07:51.424816 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424808 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-web" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424823 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="init-config-reloader" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424828 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="init-config-reloader" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424834 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-thanos" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424839 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-thanos" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424789 2558 scope.go:117] "RemoveContainer" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424923 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="prometheus" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424935 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="thanos-sidecar" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424944 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="config-reloader" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424950 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-thanos" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424956 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy-web" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.424965 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e013d54d-30c0-4c25-a444-82abb7399277" containerName="kube-rbac-proxy" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425081 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} err="failed to get container status \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": rpc error: code = NotFound desc = could not find container \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": container with ID starting with 60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed not found: ID does not exist" Jul 14 17:07:51.425144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425094 2558 scope.go:117] "RemoveContainer" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.425649 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425282 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} err="failed to get container status \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": rpc error: code = NotFound desc = could not find container \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": container with ID starting with 950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f not found: ID does not exist" Jul 14 17:07:51.425649 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425298 2558 scope.go:117] "RemoveContainer" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.425649 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425485 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} err="failed to get container status \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": rpc error: code = NotFound desc = could not find container \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": container with ID starting with 179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134 not found: ID does not exist" Jul 14 17:07:51.425649 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425497 2558 scope.go:117] "RemoveContainer" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.425783 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425731 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} err="failed to get container status \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": rpc error: code = NotFound desc = could not find container \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": container with ID starting with c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957 not found: ID does not exist" Jul 14 17:07:51.425783 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425744 2558 scope.go:117] "RemoveContainer" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.425983 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425967 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} err="failed to get container status \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": rpc error: code = NotFound desc = could not find container \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": container with ID starting with 416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45 not found: ID does not exist" Jul 14 17:07:51.426020 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.425984 2558 scope.go:117] "RemoveContainer" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.426201 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426177 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} err="failed to get container status \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": rpc error: code = NotFound desc = could not find container \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": container with ID starting with 086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1 not found: ID does not exist" Jul 14 17:07:51.426201 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426199 2558 scope.go:117] "RemoveContainer" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.426406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426381 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} err="failed to get container status \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": rpc error: code = NotFound desc = could not find container \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": container with ID starting with ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b not found: ID does not exist" Jul 14 17:07:51.426452 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426406 2558 scope.go:117] "RemoveContainer" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.426599 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426583 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} err="failed to get container status \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": rpc error: code = NotFound desc = could not find container \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": container with ID starting with 60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed not found: ID does not exist" Jul 14 17:07:51.426653 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426600 2558 scope.go:117] "RemoveContainer" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.426797 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426778 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} err="failed to get container status \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": rpc error: code = NotFound desc = could not find container \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": container with ID starting with 950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f not found: ID does not exist" Jul 14 17:07:51.426857 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.426799 2558 scope.go:117] "RemoveContainer" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.427032 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427016 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} err="failed to get container status \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": rpc error: code = NotFound desc = could not find container \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": container with ID starting with 179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134 not found: ID does not exist" Jul 14 17:07:51.427075 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427033 2558 scope.go:117] "RemoveContainer" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.427259 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427231 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} err="failed to get container status \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": rpc error: code = NotFound desc = could not find container \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": container with ID starting with c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957 not found: ID does not exist" Jul 14 17:07:51.427331 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427260 2558 scope.go:117] "RemoveContainer" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.427495 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427476 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} err="failed to get container status \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": rpc error: code = NotFound desc = could not find container \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": container with ID starting with 416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45 not found: ID does not exist" Jul 14 17:07:51.427560 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427497 2558 scope.go:117] "RemoveContainer" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.427729 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427709 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} err="failed to get container status \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": rpc error: code = NotFound desc = could not find container \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": container with ID starting with 086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1 not found: ID does not exist" Jul 14 17:07:51.427794 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427729 2558 scope.go:117] "RemoveContainer" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.427942 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427924 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} err="failed to get container status \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": rpc error: code = NotFound desc = could not find container \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": container with ID starting with ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b not found: ID does not exist" Jul 14 17:07:51.427991 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.427942 2558 scope.go:117] "RemoveContainer" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.428153 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428132 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} err="failed to get container status \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": rpc error: code = NotFound desc = could not find container \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": container with ID starting with 60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed not found: ID does not exist" Jul 14 17:07:51.428153 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428152 2558 scope.go:117] "RemoveContainer" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.428372 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428356 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} err="failed to get container status \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": rpc error: code = NotFound desc = could not find container \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": container with ID starting with 950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f not found: ID does not exist" Jul 14 17:07:51.428412 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428377 2558 scope.go:117] "RemoveContainer" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.428624 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428602 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} err="failed to get container status \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": rpc error: code = NotFound desc = could not find container \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": container with ID starting with 179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134 not found: ID does not exist" Jul 14 17:07:51.428702 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428625 2558 scope.go:117] "RemoveContainer" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.428857 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428839 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} err="failed to get container status \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": rpc error: code = NotFound desc = could not find container \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": container with ID starting with c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957 not found: ID does not exist" Jul 14 17:07:51.428903 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.428859 2558 scope.go:117] "RemoveContainer" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.429069 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429053 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} err="failed to get container status \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": rpc error: code = NotFound desc = could not find container \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": container with ID starting with 416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45 not found: ID does not exist" Jul 14 17:07:51.429119 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429070 2558 scope.go:117] "RemoveContainer" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.429277 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429256 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} err="failed to get container status \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": rpc error: code = NotFound desc = could not find container \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": container with ID starting with 086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1 not found: ID does not exist" Jul 14 17:07:51.429277 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429277 2558 scope.go:117] "RemoveContainer" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.429468 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429452 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} err="failed to get container status \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": rpc error: code = NotFound desc = could not find container \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": container with ID starting with ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b not found: ID does not exist" Jul 14 17:07:51.429525 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429469 2558 scope.go:117] "RemoveContainer" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.429775 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429757 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} err="failed to get container status \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": rpc error: code = NotFound desc = could not find container \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": container with ID starting with 60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed not found: ID does not exist" Jul 14 17:07:51.429856 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.429775 2558 scope.go:117] "RemoveContainer" containerID="950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f" Jul 14 17:07:51.430353 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.430332 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f"} err="failed to get container status \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": rpc error: code = NotFound desc = could not find container \"950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f\": container with ID starting with 950209427f01ac158627d04195911c7292cbc5a300a353cd2eb4fe1bfade776f not found: ID does not exist" Jul 14 17:07:51.430407 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.430355 2558 scope.go:117] "RemoveContainer" containerID="179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134" Jul 14 17:07:51.430597 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.430579 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134"} err="failed to get container status \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": rpc error: code = NotFound desc = could not find container \"179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134\": container with ID starting with 179ffb3d4d776d37234f6a31e7c75de33518a7051fb8ecd53e5fd91510ace134 not found: ID does not exist" Jul 14 17:07:51.430660 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.430597 2558 scope.go:117] "RemoveContainer" containerID="c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957" Jul 14 17:07:51.430712 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.430674 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.431063 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.430862 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957"} err="failed to get container status \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": rpc error: code = NotFound desc = could not find container \"c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957\": container with ID starting with c6e8bff696080df8a93b7509d0106b66a496537a097c28361ab73d3a8806d957 not found: ID does not exist" Jul 14 17:07:51.431063 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.430883 2558 scope.go:117] "RemoveContainer" containerID="416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45" Jul 14 17:07:51.431722 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.431436 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45"} err="failed to get container status \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": rpc error: code = NotFound desc = could not find container \"416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45\": container with ID starting with 416174ac738c981d9107ecddba7ee3836ffb8c71392065e651bac841ac714b45 not found: ID does not exist" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.431912 2558 scope.go:117] "RemoveContainer" containerID="086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.434108 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.434383 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.434397 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.434903 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.435148 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-znklr\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.435431 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.435507 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.435663 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.435662 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 17:07:51.435823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.435771 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 17:07:51.436476 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.436179 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1"} err="failed to get container status \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": rpc error: code = NotFound desc = could not find container \"086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1\": container with ID starting with 086be7afa121e6d8858896069fc12d0a7c78972928155076d2c09ba7c3b198a1 not found: ID does not exist" Jul 14 17:07:51.436476 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.436218 2558 scope.go:117] "RemoveContainer" containerID="ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b" Jul 14 17:07:51.436476 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.436299 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-64dctuuslokoi\"" Jul 14 17:07:51.436476 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.436330 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 17:07:51.436966 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.436937 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b"} err="failed to get container status \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": rpc error: code = NotFound desc = could not find container \"ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b\": container with ID starting with ba79ca5b0fef29fe22d5bef9f7cc4b18b1b79a45136e0551731686ad9490c94b not found: ID does not exist" Jul 14 17:07:51.437121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.436969 2558 scope.go:117] "RemoveContainer" containerID="60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed" Jul 14 17:07:51.437121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.437050 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 17:07:51.437121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.437061 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 17:07:51.438268 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.438088 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed"} err="failed to get container status \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": rpc error: code = NotFound desc = could not find container \"60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed\": container with ID starting with 60e18ff81f2da29d90d9eb3634578a354e8ce302493dda3b475bacaaa42577ed not found: ID does not exist" Jul 14 17:07:51.439287 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.438411 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 17:07:51.440704 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.440685 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 17:07:51.442556 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.442516 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 17:07:51.448178 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.448153 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:07:51.491731 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.491699 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-config\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.491910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.491735 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.491910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.491766 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.491910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.491818 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.491910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.491892 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492106 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.491938 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492106 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492016 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492106 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492087 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492255 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492116 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492255 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492191 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492255 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492248 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492393 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492281 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492393 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492313 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2hmd\" (UniqueName: \"kubernetes.io/projected/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-kube-api-access-s2hmd\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492486 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492466 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492536 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492504 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492618 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492691 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492673 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492790 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492773 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492863 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492808 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.492863 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.492835 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594303 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594264 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594320 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594343 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594366 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594385 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2hmd\" (UniqueName: \"kubernetes.io/projected/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-kube-api-access-s2hmd\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594405 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594422 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.594489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594449 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595051 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.594497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595351 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595322 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595448 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595387 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595448 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595397 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595448 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595426 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595639 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595454 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595639 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595492 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-config\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595639 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595516 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595639 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595568 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595639 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595596 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595639 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595624 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595928 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595652 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595928 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595656 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595928 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.595928 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.595739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.596623 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.596283 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.597833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.597805 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.597833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.597818 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.597981 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.597915 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.598044 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.597992 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.598099 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.598052 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.598247 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.598226 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.598821 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.598573 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.598821 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.598627 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.598821 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.598673 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.599018 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.598880 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.599277 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.599252 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.600113 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.600088 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.600266 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.600249 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.600552 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.600522 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-config\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.601103 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.601085 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.603182 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.603163 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2hmd\" (UniqueName: \"kubernetes.io/projected/3a5e27b3-26bf-4c65-9f93-70647dcff7f8-kube-api-access-s2hmd\") pod \"prometheus-k8s-0\" (UID: \"3a5e27b3-26bf-4c65-9f93-70647dcff7f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.745777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.745745 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:07:51.898597 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:51.898566 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:07:51.900374 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:07:51.900349 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5e27b3_26bf_4c65_9f93_70647dcff7f8.slice/crio-9c93400d12ace48dacdb6b465e9cf33ede0ba79295b0f7f8868e4d2c396311ed WatchSource:0}: Error finding container 9c93400d12ace48dacdb6b465e9cf33ede0ba79295b0f7f8868e4d2c396311ed: Status 404 returned error can't find the container with id 9c93400d12ace48dacdb6b465e9cf33ede0ba79295b0f7f8868e4d2c396311ed Jul 14 17:07:52.367058 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:52.367023 2558 generic.go:356] "Generic (PLEG): container finished" podID="3a5e27b3-26bf-4c65-9f93-70647dcff7f8" containerID="b186636a62c2f6249d2dada24c6bea306c4ea5b6fd80aeead1e8ef4a0bd083fe" exitCode=0 Jul 14 17:07:52.367528 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:52.367112 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerDied","Data":"b186636a62c2f6249d2dada24c6bea306c4ea5b6fd80aeead1e8ef4a0bd083fe"} Jul 14 17:07:52.367528 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:52.367199 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerStarted","Data":"9c93400d12ace48dacdb6b465e9cf33ede0ba79295b0f7f8868e4d2c396311ed"} Jul 14 17:07:53.381264 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.381235 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e013d54d-30c0-4c25-a444-82abb7399277" path="/var/lib/kubelet/pods/e013d54d-30c0-4c25-a444-82abb7399277/volumes" Jul 14 17:07:53.381797 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.381777 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerStarted","Data":"29ab570c42cc2f42588e2919a67f5b3ed9080a9c37e2c349fc1d3be037cffc84"} Jul 14 17:07:53.381853 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.381842 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerStarted","Data":"18aee8cbb8933248f9b17bf84271cad3cd418143c8c45247e33797b68e92d505"} Jul 14 17:07:53.381903 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.381872 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerStarted","Data":"0ef5a17dd661e31cc8d652fe3c0cfb99e4108b9b17dbb24b6f0a4e6171db4911"} Jul 14 17:07:53.381903 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.381895 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerStarted","Data":"3afefb06ae7235cf01e58053a8ff9e5cdc49cf0de19977820dbaa405af82c515"} Jul 14 17:07:53.381960 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.381918 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerStarted","Data":"518c4cf39cc10e90fea4ea5dd0371b054f19a48225c85da465e3e0438d54ff29"} Jul 14 17:07:53.381960 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.381952 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3a5e27b3-26bf-4c65-9f93-70647dcff7f8","Type":"ContainerStarted","Data":"9e1b5800b416eb6078ea1ac72977f21260a35a9ea10e9d8fa277a8f081a386fe"} Jul 14 17:07:53.414647 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:53.414575 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.395707733 podStartE2EDuration="2.414535334s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.827601ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:07:53.408168919 +0000 UTC m=+175.977579693" watchObservedRunningTime="2026-07-14 17:07:53.414535334 +0000 UTC m=+175.983946106" Jul 14 17:07:56.746430 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:07:56.746397 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:08:51.746749 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:08:51.746702 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:08:51.763699 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:08:51.763666 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:08:52.672791 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:08:52.672760 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:09:59.008525 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:09:59.008491 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:09:59.009131 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:09:59.009111 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:09:59.011288 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:09:59.011265 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:09:59.012202 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:09:59.012179 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:09:59.019936 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:09:59.019909 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 17:10:07.960444 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.960407 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77"] Jul 14 17:10:07.964008 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.963986 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:07.967718 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.967694 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 17:10:07.967854 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.967722 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 17:10:07.967854 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.967790 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 17:10:07.968011 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.967998 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fv67k\"" Jul 14 17:10:07.968073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.968060 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 17:10:07.978534 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.978507 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77"] Jul 14 17:10:07.984626 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.984594 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzjvc\" (UniqueName: \"kubernetes.io/projected/4b907c15-81f1-454a-b936-6d5ae0d886e5-kube-api-access-nzjvc\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:07.984747 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.984658 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b907c15-81f1-454a-b936-6d5ae0d886e5-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:07.984747 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.984727 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b907c15-81f1-454a-b936-6d5ae0d886e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:07.984840 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.984777 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b907c15-81f1-454a-b936-6d5ae0d886e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:07.984885 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:07.984835 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b907c15-81f1-454a-b936-6d5ae0d886e5-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.085487 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.085446 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b907c15-81f1-454a-b936-6d5ae0d886e5-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.085741 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.085504 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b907c15-81f1-454a-b936-6d5ae0d886e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.085741 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.085595 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b907c15-81f1-454a-b936-6d5ae0d886e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.085741 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.085631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b907c15-81f1-454a-b936-6d5ae0d886e5-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.085741 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.085686 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzjvc\" (UniqueName: \"kubernetes.io/projected/4b907c15-81f1-454a-b936-6d5ae0d886e5-kube-api-access-nzjvc\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.086104 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.086069 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b907c15-81f1-454a-b936-6d5ae0d886e5-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.087843 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.087823 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b907c15-81f1-454a-b936-6d5ae0d886e5-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.088164 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.088144 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b907c15-81f1-454a-b936-6d5ae0d886e5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.088208 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.088171 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b907c15-81f1-454a-b936-6d5ae0d886e5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.094708 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.094685 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzjvc\" (UniqueName: \"kubernetes.io/projected/4b907c15-81f1-454a-b936-6d5ae0d886e5-kube-api-access-nzjvc\") pod \"opendatahub-operator-controller-manager-7655f47d9d-xpk77\" (UID: \"4b907c15-81f1-454a-b936-6d5ae0d886e5\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.274263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.274231 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:08.424966 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.424933 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77"] Jul 14 17:10:08.428297 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:10:08.428267 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b907c15_81f1_454a_b936_6d5ae0d886e5.slice/crio-a7ab3468351730a97004c17627364566e6935db4e10a9646c9f1c6544bcc61b9 WatchSource:0}: Error finding container a7ab3468351730a97004c17627364566e6935db4e10a9646c9f1c6544bcc61b9: Status 404 returned error can't find the container with id a7ab3468351730a97004c17627364566e6935db4e10a9646c9f1c6544bcc61b9 Jul 14 17:10:08.430164 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.430147 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:10:08.999237 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:08.999200 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" event={"ID":"4b907c15-81f1-454a-b936-6d5ae0d886e5","Type":"ContainerStarted","Data":"a7ab3468351730a97004c17627364566e6935db4e10a9646c9f1c6544bcc61b9"} Jul 14 17:10:12.019531 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:12.019494 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" event={"ID":"4b907c15-81f1-454a-b936-6d5ae0d886e5","Type":"ContainerStarted","Data":"23d1a58710f93f9fb62a522c0cdf31eb0fb4c7bbd3efe6303924dfc58e5f9605"} Jul 14 17:10:13.027763 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:13.027664 2558 generic.go:356] "Generic (PLEG): container finished" podID="4b907c15-81f1-454a-b936-6d5ae0d886e5" containerID="23d1a58710f93f9fb62a522c0cdf31eb0fb4c7bbd3efe6303924dfc58e5f9605" exitCode=0 Jul 14 17:10:13.028121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:13.027753 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" event={"ID":"4b907c15-81f1-454a-b936-6d5ae0d886e5","Type":"ContainerDied","Data":"23d1a58710f93f9fb62a522c0cdf31eb0fb4c7bbd3efe6303924dfc58e5f9605"} Jul 14 17:10:14.034856 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:14.034819 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" event={"ID":"4b907c15-81f1-454a-b936-6d5ae0d886e5","Type":"ContainerStarted","Data":"980648d2314ce4991ba6e92423d6babaf713a57d83b64930ad25623005ae72b3"} Jul 14 17:10:14.035332 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:14.034939 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:14.088341 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:14.088283 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" podStartSLOduration=2.462821563 podStartE2EDuration="7.088265196s" totalImagesPullingTime="3.533238908s" totalInitContainerRuntime="1.092204725s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:14.084688654 +0000 UTC m=+316.654099444" watchObservedRunningTime="2026-07-14 17:10:14.088265196 +0000 UTC m=+316.657675966" Jul 14 17:10:25.042786 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:25.042757 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-xpk77" Jul 14 17:10:35.251051 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.251016 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s"] Jul 14 17:10:35.257643 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.257609 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.261098 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.261074 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 17:10:35.261421 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.261403 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 17:10:35.261477 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.261453 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6wdsg\"" Jul 14 17:10:35.261589 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.261573 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 17:10:35.266122 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.265863 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 17:10:35.276359 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.276317 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s"] Jul 14 17:10:35.327614 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.327570 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sn6x\" (UniqueName: \"kubernetes.io/projected/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-kube-api-access-2sn6x\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.327614 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.327612 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.327829 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.327682 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.327829 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.327717 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.327829 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.327793 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.327829 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.327827 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.327955 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.327880 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.429308 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.429267 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.429512 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.429321 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.429512 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.429346 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.429512 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.429389 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sn6x\" (UniqueName: \"kubernetes.io/projected/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-kube-api-access-2sn6x\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.429512 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.429404 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.429512 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.429437 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.429819 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.429620 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.430242 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.430214 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.432019 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.431994 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.432019 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.432016 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.432196 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.432174 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.432256 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.432207 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.443289 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.443261 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.443467 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.443447 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sn6x\" (UniqueName: \"kubernetes.io/projected/3920de5e-8ecc-4746-b3e5-edb2d9d3d688-kube-api-access-2sn6x\") pod \"istiod-openshift-gateway-94bb8fbfd-vsq7s\" (UID: \"3920de5e-8ecc-4746-b3e5-edb2d9d3d688\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.469978 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.469939 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm"] Jul 14 17:10:35.472992 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.472965 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.475979 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.475955 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 17:10:35.476141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.476047 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jxjns\"" Jul 14 17:10:35.476141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.476052 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 17:10:35.476468 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.476449 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:10:35.476468 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.476461 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 17:10:35.476640 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.476576 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 17:10:35.483336 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.483307 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm"] Jul 14 17:10:35.530823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.530787 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-manager-config\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.531028 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.530828 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htjqb\" (UniqueName: \"kubernetes.io/projected/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-kube-api-access-htjqb\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.531028 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.530872 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-metrics-cert\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.531028 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.530965 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-cert\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.568900 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.568861 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:35.631705 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.631660 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-cert\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.631890 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.631808 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-manager-config\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.631890 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.631839 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htjqb\" (UniqueName: \"kubernetes.io/projected/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-kube-api-access-htjqb\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.632015 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.631940 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-metrics-cert\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.632995 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.632937 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-manager-config\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.636292 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.635071 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-metrics-cert\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.636292 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.635192 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-cert\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.641954 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.641926 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htjqb\" (UniqueName: \"kubernetes.io/projected/4b90d5c7-ee54-41cc-9ef5-9c07023112eb-kube-api-access-htjqb\") pod \"lws-controller-manager-5684c55548-5xmzm\" (UID: \"4b90d5c7-ee54-41cc-9ef5-9c07023112eb\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.733629 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.733602 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s"] Jul 14 17:10:35.735490 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:10:35.735461 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3920de5e_8ecc_4746_b3e5_edb2d9d3d688.slice/crio-f76fd351bd8ef086d7a0d592bbb3c4b93b654adeaf5059c6eb28789d9fa05891 WatchSource:0}: Error finding container f76fd351bd8ef086d7a0d592bbb3c4b93b654adeaf5059c6eb28789d9fa05891: Status 404 returned error can't find the container with id f76fd351bd8ef086d7a0d592bbb3c4b93b654adeaf5059c6eb28789d9fa05891 Jul 14 17:10:35.785263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.785172 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:35.955669 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:35.955633 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm"] Jul 14 17:10:35.959356 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:10:35.959290 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b90d5c7_ee54_41cc_9ef5_9c07023112eb.slice/crio-42052c8d44d4007bb801dec6ff00970e91f05b311bfdb56c821a8b5c393a8ce9 WatchSource:0}: Error finding container 42052c8d44d4007bb801dec6ff00970e91f05b311bfdb56c821a8b5c393a8ce9: Status 404 returned error can't find the container with id 42052c8d44d4007bb801dec6ff00970e91f05b311bfdb56c821a8b5c393a8ce9 Jul 14 17:10:36.144030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:36.143933 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" event={"ID":"4b90d5c7-ee54-41cc-9ef5-9c07023112eb","Type":"ContainerStarted","Data":"42052c8d44d4007bb801dec6ff00970e91f05b311bfdb56c821a8b5c393a8ce9"} Jul 14 17:10:36.145208 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:36.145180 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" event={"ID":"3920de5e-8ecc-4746-b3e5-edb2d9d3d688","Type":"ContainerStarted","Data":"f76fd351bd8ef086d7a0d592bbb3c4b93b654adeaf5059c6eb28789d9fa05891"} Jul 14 17:10:39.548927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:39.548884 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:10:39.549318 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:39.548960 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:10:40.194105 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.194064 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" event={"ID":"4b90d5c7-ee54-41cc-9ef5-9c07023112eb","Type":"ContainerStarted","Data":"d66707a0af300b1dc6e5bb077832acd11de7f31bc4df6804abb6fbe270d65342"} Jul 14 17:10:40.194579 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.194558 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:10:40.197884 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.196107 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" event={"ID":"3920de5e-8ecc-4746-b3e5-edb2d9d3d688","Type":"ContainerStarted","Data":"9f7226fba74011c3afaf90b54517d35b5878318cf1b0ab86556601bfaf0bc07b"} Jul 14 17:10:40.197884 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.197234 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:40.198906 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.198877 2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-vsq7s container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 17:10:40.199087 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.199067 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" podUID="3920de5e-8ecc-4746-b3e5-edb2d9d3d688" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:10:40.216176 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.216109 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" podStartSLOduration=1.631478173 podStartE2EDuration="5.216086816s" totalImagesPullingTime="3.584608643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:40.213718198 +0000 UTC m=+342.783128998" watchObservedRunningTime="2026-07-14 17:10:40.216086816 +0000 UTC m=+342.785497568" Jul 14 17:10:40.246626 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:40.246576 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" podStartSLOduration=1.435527787 podStartE2EDuration="5.246558955s" totalImagesPullingTime="3.811031168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:40.244112606 +0000 UTC m=+342.813523411" watchObservedRunningTime="2026-07-14 17:10:40.246558955 +0000 UTC m=+342.815969724" Jul 14 17:10:41.201665 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:41.201639 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vsq7s" Jul 14 17:10:52.208184 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:10:52.208099 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5684c55548-5xmzm" Jul 14 17:12:18.992854 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:18.992812 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2"] Jul 14 17:12:18.995836 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:18.995811 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:18.998505 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:18.998475 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 17:12:18.998660 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:18.998475 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mlgp9\"" Jul 14 17:12:18.998660 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:18.998475 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 17:12:19.010594 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.010563 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2"] Jul 14 17:12:19.127159 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.127116 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f74525e1-dd29-4335-91bb-80764a50a8b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-slps2\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:19.127353 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.127223 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4btmh\" (UniqueName: \"kubernetes.io/projected/f74525e1-dd29-4335-91bb-80764a50a8b9-kube-api-access-4btmh\") pod \"kuadrant-operator-controller-manager-86878c769f-slps2\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:19.228585 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.228461 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4btmh\" (UniqueName: \"kubernetes.io/projected/f74525e1-dd29-4335-91bb-80764a50a8b9-kube-api-access-4btmh\") pod \"kuadrant-operator-controller-manager-86878c769f-slps2\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:19.228773 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.228614 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f74525e1-dd29-4335-91bb-80764a50a8b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-slps2\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:19.228989 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.228969 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f74525e1-dd29-4335-91bb-80764a50a8b9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-slps2\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:19.239851 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.239816 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4btmh\" (UniqueName: \"kubernetes.io/projected/f74525e1-dd29-4335-91bb-80764a50a8b9-kube-api-access-4btmh\") pod \"kuadrant-operator-controller-manager-86878c769f-slps2\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:19.306907 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.306856 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:19.471979 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.471941 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2"] Jul 14 17:12:19.477706 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:12:19.477675 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf74525e1_dd29_4335_91bb_80764a50a8b9.slice/crio-399655ab0f48e1fc98d8e037b8e653b741a74a1fac41109bb79a0af2101d7fb9 WatchSource:0}: Error finding container 399655ab0f48e1fc98d8e037b8e653b741a74a1fac41109bb79a0af2101d7fb9: Status 404 returned error can't find the container with id 399655ab0f48e1fc98d8e037b8e653b741a74a1fac41109bb79a0af2101d7fb9 Jul 14 17:12:19.697925 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:19.697831 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" event={"ID":"f74525e1-dd29-4335-91bb-80764a50a8b9","Type":"ContainerStarted","Data":"399655ab0f48e1fc98d8e037b8e653b741a74a1fac41109bb79a0af2101d7fb9"} Jul 14 17:12:25.738453 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:25.738405 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" event={"ID":"f74525e1-dd29-4335-91bb-80764a50a8b9","Type":"ContainerStarted","Data":"1c69544e8eb202efcd09dc5d2b920f6e78f8b26bdf67120172309f8cde136d14"} Jul 14 17:12:25.738942 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:25.738515 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:12:25.763797 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:25.763740 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" podStartSLOduration=2.238076107 podStartE2EDuration="7.763722179s" totalImagesPullingTime="5.525646072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:12:25.762244739 +0000 UTC m=+448.331655540" watchObservedRunningTime="2026-07-14 17:12:25.763722179 +0000 UTC m=+448.333132950" Jul 14 17:12:36.745454 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:12:36.745421 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:13:20.677371 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.677327 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z47b9"] Jul 14 17:13:20.680179 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.680153 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:20.682917 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.682896 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 14 17:13:20.682917 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.682916 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 17:13:20.683115 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.682918 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-l78rz\"" Jul 14 17:13:20.683115 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.682973 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 17:13:20.689690 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.689664 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z47b9"] Jul 14 17:13:20.828970 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.828937 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ztbz\" (UniqueName: \"kubernetes.io/projected/f8883ec1-f2a6-4982-b254-1b1be580d9c5-kube-api-access-5ztbz\") pod \"postgres-69fb4594cf-z47b9\" (UID: \"f8883ec1-f2a6-4982-b254-1b1be580d9c5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:20.829157 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.829098 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f8883ec1-f2a6-4982-b254-1b1be580d9c5-data\") pod \"postgres-69fb4594cf-z47b9\" (UID: \"f8883ec1-f2a6-4982-b254-1b1be580d9c5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:20.930056 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.929966 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f8883ec1-f2a6-4982-b254-1b1be580d9c5-data\") pod \"postgres-69fb4594cf-z47b9\" (UID: \"f8883ec1-f2a6-4982-b254-1b1be580d9c5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:20.930056 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.930003 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ztbz\" (UniqueName: \"kubernetes.io/projected/f8883ec1-f2a6-4982-b254-1b1be580d9c5-kube-api-access-5ztbz\") pod \"postgres-69fb4594cf-z47b9\" (UID: \"f8883ec1-f2a6-4982-b254-1b1be580d9c5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:20.930422 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.930393 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f8883ec1-f2a6-4982-b254-1b1be580d9c5-data\") pod \"postgres-69fb4594cf-z47b9\" (UID: \"f8883ec1-f2a6-4982-b254-1b1be580d9c5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:20.938995 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.938966 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ztbz\" (UniqueName: \"kubernetes.io/projected/f8883ec1-f2a6-4982-b254-1b1be580d9c5-kube-api-access-5ztbz\") pod \"postgres-69fb4594cf-z47b9\" (UID: \"f8883ec1-f2a6-4982-b254-1b1be580d9c5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:20.991768 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:20.991728 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:21.133361 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:21.133335 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z47b9"] Jul 14 17:13:21.135517 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:13:21.135486 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8883ec1_f2a6_4982_b254_1b1be580d9c5.slice/crio-0c8000bd60a6725a31700ccec1d1848b68feb911e0ab94b208e9fb8ab9965ada WatchSource:0}: Error finding container 0c8000bd60a6725a31700ccec1d1848b68feb911e0ab94b208e9fb8ab9965ada: Status 404 returned error can't find the container with id 0c8000bd60a6725a31700ccec1d1848b68feb911e0ab94b208e9fb8ab9965ada Jul 14 17:13:22.019110 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:22.019065 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" event={"ID":"f8883ec1-f2a6-4982-b254-1b1be580d9c5","Type":"ContainerStarted","Data":"0c8000bd60a6725a31700ccec1d1848b68feb911e0ab94b208e9fb8ab9965ada"} Jul 14 17:13:27.051652 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:27.051602 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" event={"ID":"f8883ec1-f2a6-4982-b254-1b1be580d9c5","Type":"ContainerStarted","Data":"959d8ad09274646658e36846c18eabc413c534bd1977a91f4d4dd199c2714c59"} Jul 14 17:13:27.052130 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:27.051720 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:13:27.070370 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:27.070306 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" podStartSLOduration=1.771116192 podStartE2EDuration="7.070287216s" totalImagesPullingTime="5.299171024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:27.068289752 +0000 UTC m=+509.637700523" watchObservedRunningTime="2026-07-14 17:13:27.070287216 +0000 UTC m=+509.639697987" Jul 14 17:13:33.087339 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:13:33.087310 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z47b9" Jul 14 17:14:53.755673 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.755632 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl"] Jul 14 17:14:53.759687 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.759655 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.764743 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.764713 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xt9hh\"" Jul 14 17:14:53.764743 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.764713 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 17:14:53.764961 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.764759 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 17:14:53.764961 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.764722 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 17:14:53.772645 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.772618 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl"] Jul 14 17:14:53.827085 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.827044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.827085 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.827079 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65659adf-b41f-44f7-a472-35898be98e61-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.827330 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.827132 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.827330 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.827215 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.827330 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.827251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.827431 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.827335 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r5vb\" (UniqueName: \"kubernetes.io/projected/65659adf-b41f-44f7-a472-35898be98e61-kube-api-access-7r5vb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.928420 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928376 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.928420 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928418 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65659adf-b41f-44f7-a472-35898be98e61-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.928681 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928460 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.928681 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928513 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.928681 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928565 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.928681 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r5vb\" (UniqueName: \"kubernetes.io/projected/65659adf-b41f-44f7-a472-35898be98e61-kube-api-access-7r5vb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.928946 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928924 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.929019 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.928998 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.929076 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.929042 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.930966 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.930936 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65659adf-b41f-44f7-a472-35898be98e61-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.931224 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.931203 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65659adf-b41f-44f7-a472-35898be98e61-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:53.938890 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:53.938854 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r5vb\" (UniqueName: \"kubernetes.io/projected/65659adf-b41f-44f7-a472-35898be98e61-kube-api-access-7r5vb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jlprl\" (UID: \"65659adf-b41f-44f7-a472-35898be98e61\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:54.070924 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:54.070875 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:14:54.217730 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:54.217698 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl"] Jul 14 17:14:54.222604 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:14:54.219200 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65659adf_b41f_44f7_a472_35898be98e61.slice/crio-b8005a0fd8226f8d1c56a9821e8baa16fdc7ec090616a1a02a599156f299987c WatchSource:0}: Error finding container b8005a0fd8226f8d1c56a9821e8baa16fdc7ec090616a1a02a599156f299987c: Status 404 returned error can't find the container with id b8005a0fd8226f8d1c56a9821e8baa16fdc7ec090616a1a02a599156f299987c Jul 14 17:14:54.519359 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:54.519279 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" event={"ID":"65659adf-b41f-44f7-a472-35898be98e61","Type":"ContainerStarted","Data":"b8005a0fd8226f8d1c56a9821e8baa16fdc7ec090616a1a02a599156f299987c"} Jul 14 17:14:55.526367 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:55.526273 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" event={"ID":"65659adf-b41f-44f7-a472-35898be98e61","Type":"ContainerStarted","Data":"280bd3a1a138f09cf655075c5540a4f03adb8fbfa83d1af8df5664d0511a60f1"} Jul 14 17:14:55.547953 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:55.547901 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" podStartSLOduration=1.49537907 podStartE2EDuration="2.547883011s" totalImagesPullingTime="1.052503941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:14:55.546684215 +0000 UTC m=+598.116094987" watchObservedRunningTime="2026-07-14 17:14:55.547883011 +0000 UTC m=+598.117293782" Jul 14 17:14:59.045961 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:59.045928 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:14:59.046398 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:59.045981 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:14:59.048282 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:59.048262 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:14:59.048427 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:14:59.048263 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:15:04.071617 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:15:04.071574 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:15:04.072025 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:15:04.071650 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:15:04.084229 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:15:04.084200 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:15:04.584234 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:15:04.584203 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jlprl" Jul 14 17:19:59.085883 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:19:59.085781 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:19:59.086686 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:19:59.086667 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:19:59.088475 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:19:59.088451 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:19:59.089221 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:19:59.089198 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:23:09.326917 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:09.326875 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2"] Jul 14 17:23:09.327534 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:09.327124 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" podUID="f74525e1-dd29-4335-91bb-80764a50a8b9" containerName="manager" containerID="cri-o://1c69544e8eb202efcd09dc5d2b920f6e78f8b26bdf67120172309f8cde136d14" gracePeriod=10 Jul 14 17:23:14.164486 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.164445 2558 generic.go:356] "Generic (PLEG): container finished" podID="f74525e1-dd29-4335-91bb-80764a50a8b9" containerID="1c69544e8eb202efcd09dc5d2b920f6e78f8b26bdf67120172309f8cde136d14" exitCode=0 Jul 14 17:23:14.165021 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.164517 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" event={"ID":"f74525e1-dd29-4335-91bb-80764a50a8b9","Type":"ContainerDied","Data":"1c69544e8eb202efcd09dc5d2b920f6e78f8b26bdf67120172309f8cde136d14"} Jul 14 17:23:14.297159 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.297123 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:23:14.434073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.433946 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4btmh\" (UniqueName: \"kubernetes.io/projected/f74525e1-dd29-4335-91bb-80764a50a8b9-kube-api-access-4btmh\") pod \"f74525e1-dd29-4335-91bb-80764a50a8b9\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " Jul 14 17:23:14.434073 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.434013 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f74525e1-dd29-4335-91bb-80764a50a8b9-extensions-socket-volume\") pod \"f74525e1-dd29-4335-91bb-80764a50a8b9\" (UID: \"f74525e1-dd29-4335-91bb-80764a50a8b9\") " Jul 14 17:23:14.434387 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.434360 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f74525e1-dd29-4335-91bb-80764a50a8b9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f74525e1-dd29-4335-91bb-80764a50a8b9" (UID: "f74525e1-dd29-4335-91bb-80764a50a8b9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:23:14.436553 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.436500 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f74525e1-dd29-4335-91bb-80764a50a8b9-kube-api-access-4btmh" (OuterVolumeSpecName: "kube-api-access-4btmh") pod "f74525e1-dd29-4335-91bb-80764a50a8b9" (UID: "f74525e1-dd29-4335-91bb-80764a50a8b9"). InnerVolumeSpecName "kube-api-access-4btmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:23:14.535818 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.535772 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4btmh\" (UniqueName: \"kubernetes.io/projected/f74525e1-dd29-4335-91bb-80764a50a8b9-kube-api-access-4btmh\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:23:14.535818 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:14.535806 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f74525e1-dd29-4335-91bb-80764a50a8b9-extensions-socket-volume\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:23:15.170695 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:15.170665 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" Jul 14 17:23:15.170695 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:15.170674 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2" event={"ID":"f74525e1-dd29-4335-91bb-80764a50a8b9","Type":"ContainerDied","Data":"399655ab0f48e1fc98d8e037b8e653b741a74a1fac41109bb79a0af2101d7fb9"} Jul 14 17:23:15.171336 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:15.170739 2558 scope.go:117] "RemoveContainer" containerID="1c69544e8eb202efcd09dc5d2b920f6e78f8b26bdf67120172309f8cde136d14" Jul 14 17:23:15.230605 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:15.230531 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2"] Jul 14 17:23:15.232581 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:15.232556 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-slps2"] Jul 14 17:23:15.382354 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:23:15.382319 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f74525e1-dd29-4335-91bb-80764a50a8b9" path="/var/lib/kubelet/pods/f74525e1-dd29-4335-91bb-80764a50a8b9/volumes" Jul 14 17:24:10.470929 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.470890 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5"] Jul 14 17:24:10.471419 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.471384 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f74525e1-dd29-4335-91bb-80764a50a8b9" containerName="manager" Jul 14 17:24:10.471419 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.471400 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f74525e1-dd29-4335-91bb-80764a50a8b9" containerName="manager" Jul 14 17:24:10.471501 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.471464 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f74525e1-dd29-4335-91bb-80764a50a8b9" containerName="manager" Jul 14 17:24:10.475833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.475801 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.480531 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.480499 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 17:24:10.480721 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.480511 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mlgp9\"" Jul 14 17:24:10.480855 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.480836 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 17:24:10.493911 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.493878 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5"] Jul 14 17:24:10.596594 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.596550 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16670199-c56b-49bf-94c9-62ead84b2332-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7x6p5\" (UID: \"16670199-c56b-49bf-94c9-62ead84b2332\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.596774 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.596649 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtz7\" (UniqueName: \"kubernetes.io/projected/16670199-c56b-49bf-94c9-62ead84b2332-kube-api-access-wbtz7\") pod \"kuadrant-operator-controller-manager-86878c769f-7x6p5\" (UID: \"16670199-c56b-49bf-94c9-62ead84b2332\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.698048 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.698004 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16670199-c56b-49bf-94c9-62ead84b2332-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7x6p5\" (UID: \"16670199-c56b-49bf-94c9-62ead84b2332\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.698251 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.698067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtz7\" (UniqueName: \"kubernetes.io/projected/16670199-c56b-49bf-94c9-62ead84b2332-kube-api-access-wbtz7\") pod \"kuadrant-operator-controller-manager-86878c769f-7x6p5\" (UID: \"16670199-c56b-49bf-94c9-62ead84b2332\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.698488 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.698460 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16670199-c56b-49bf-94c9-62ead84b2332-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7x6p5\" (UID: \"16670199-c56b-49bf-94c9-62ead84b2332\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.708326 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.708297 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtz7\" (UniqueName: \"kubernetes.io/projected/16670199-c56b-49bf-94c9-62ead84b2332-kube-api-access-wbtz7\") pod \"kuadrant-operator-controller-manager-86878c769f-7x6p5\" (UID: \"16670199-c56b-49bf-94c9-62ead84b2332\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.787149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.787095 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:10.943406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:10.943375 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5"] Jul 14 17:24:10.945336 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:24:10.945308 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16670199_c56b_49bf_94c9_62ead84b2332.slice/crio-e8638b4e2dd3221296ce48113dc3c1e8b005b4a553d0160ff56436973b8d7d89 WatchSource:0}: Error finding container e8638b4e2dd3221296ce48113dc3c1e8b005b4a553d0160ff56436973b8d7d89: Status 404 returned error can't find the container with id e8638b4e2dd3221296ce48113dc3c1e8b005b4a553d0160ff56436973b8d7d89 Jul 14 17:24:11.469256 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:11.469192 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" event={"ID":"16670199-c56b-49bf-94c9-62ead84b2332","Type":"ContainerStarted","Data":"7e6a613b729eff02df22c056432f185c8f0cb9f2701a3ee1ff15419e99f2a885"} Jul 14 17:24:11.469256 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:11.469262 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" event={"ID":"16670199-c56b-49bf-94c9-62ead84b2332","Type":"ContainerStarted","Data":"e8638b4e2dd3221296ce48113dc3c1e8b005b4a553d0160ff56436973b8d7d89"} Jul 14 17:24:11.469522 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:11.469428 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:11.493225 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:11.493166 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" podStartSLOduration=1.493147932 podStartE2EDuration="1.493147932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:24:11.49141928 +0000 UTC m=+1154.060830100" watchObservedRunningTime="2026-07-14 17:24:11.493147932 +0000 UTC m=+1154.062558751" Jul 14 17:24:22.476649 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:22.476565 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7x6p5" Jul 14 17:24:59.125192 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:59.125165 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:24:59.125781 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:59.125223 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:24:59.127471 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:59.127451 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:24:59.127597 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:24:59.127453 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:29:59.160304 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:29:59.160274 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:29:59.162932 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:29:59.161000 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:29:59.162932 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:29:59.162492 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:29:59.163249 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:29:59.163228 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:30:00.147443 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:00.147399 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx"] Jul 14 17:30:00.151137 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:00.151113 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx" Jul 14 17:30:00.154071 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:00.154044 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:30:00.161503 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:00.161480 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx" Jul 14 17:30:00.184860 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:00.184820 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx"] Jul 14 17:30:00.325093 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:00.325063 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:30:00.325516 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:00.325484 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx"] Jul 14 17:30:01.298492 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:01.298452 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx" event={"ID":"1f94cabf-8f25-40c9-8206-fac685af861d","Type":"ContainerStarted","Data":"bf83810dc4b755fac90c7666fa1e8f6518b0830ac4cdb198d3ec7420ddef2e90"} Jul 14 17:30:04.319790 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:04.319751 2558 generic.go:356] "Generic (PLEG): container finished" podID="1f94cabf-8f25-40c9-8206-fac685af861d" containerID="65a7f1b9144d5f152993ab703b455828017f21164f29a62eace49e83d58ba06b" exitCode=0 Jul 14 17:30:04.320281 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:04.319797 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx" event={"ID":"1f94cabf-8f25-40c9-8206-fac685af861d","Type":"ContainerDied","Data":"65a7f1b9144d5f152993ab703b455828017f21164f29a62eace49e83d58ba06b"} Jul 14 17:30:05.467334 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:05.467301 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx" Jul 14 17:30:06.332445 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:06.332404 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx" event={"ID":"1f94cabf-8f25-40c9-8206-fac685af861d","Type":"ContainerDied","Data":"bf83810dc4b755fac90c7666fa1e8f6518b0830ac4cdb198d3ec7420ddef2e90"} Jul 14 17:30:06.332445 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:06.332436 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734170-dw6tx" Jul 14 17:30:06.332696 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:06.332467 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf83810dc4b755fac90c7666fa1e8f6518b0830ac4cdb198d3ec7420ddef2e90" Jul 14 17:30:48.125684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.125644 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2"] Jul 14 17:30:48.126159 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.126100 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f94cabf-8f25-40c9-8206-fac685af861d" containerName="cleanup" Jul 14 17:30:48.126159 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.126112 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f94cabf-8f25-40c9-8206-fac685af861d" containerName="cleanup" Jul 14 17:30:48.126235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.126174 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f94cabf-8f25-40c9-8206-fac685af861d" containerName="cleanup" Jul 14 17:30:48.132736 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.132711 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" Jul 14 17:30:48.136453 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.136425 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:30:48.137858 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.137828 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2"] Jul 14 17:30:48.201536 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.201488 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/474e162c-9967-4bf0-a876-f073888076c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2\" (UID: \"474e162c-9967-4bf0-a876-f073888076c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" Jul 14 17:30:48.303032 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.302989 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/474e162c-9967-4bf0-a876-f073888076c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2\" (UID: \"474e162c-9967-4bf0-a876-f073888076c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" Jul 14 17:30:48.303678 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.303656 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/474e162c-9967-4bf0-a876-f073888076c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2\" (UID: \"474e162c-9967-4bf0-a876-f073888076c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" Jul 14 17:30:48.447002 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.446903 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" Jul 14 17:30:48.599164 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:48.599126 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2"] Jul 14 17:30:48.600849 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:30:48.600816 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474e162c_9967_4bf0_a876_f073888076c3.slice/crio-56a372a2899085b350e2e91697dac715244652d80013a670d3f5da6c22d7dd6d WatchSource:0}: Error finding container 56a372a2899085b350e2e91697dac715244652d80013a670d3f5da6c22d7dd6d: Status 404 returned error can't find the container with id 56a372a2899085b350e2e91697dac715244652d80013a670d3f5da6c22d7dd6d Jul 14 17:30:49.558784 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:49.558744 2558 generic.go:356] "Generic (PLEG): container finished" podID="474e162c-9967-4bf0-a876-f073888076c3" containerID="3d3ca7e925491fe799dcddb5c94809d15531ef71dc44b402ce5e5b7e3f8e22d6" exitCode=7 Jul 14 17:30:49.559235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:49.558810 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" event={"ID":"474e162c-9967-4bf0-a876-f073888076c3","Type":"ContainerDied","Data":"3d3ca7e925491fe799dcddb5c94809d15531ef71dc44b402ce5e5b7e3f8e22d6"} Jul 14 17:30:49.559235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:49.558858 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" event={"ID":"474e162c-9967-4bf0-a876-f073888076c3","Type":"ContainerStarted","Data":"56a372a2899085b350e2e91697dac715244652d80013a670d3f5da6c22d7dd6d"} Jul 14 17:30:49.559235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:49.559041 2558 scope.go:117] "RemoveContainer" containerID="3d3ca7e925491fe799dcddb5c94809d15531ef71dc44b402ce5e5b7e3f8e22d6" Jul 14 17:30:50.565247 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:50.565148 2558 generic.go:356] "Generic (PLEG): container finished" podID="474e162c-9967-4bf0-a876-f073888076c3" containerID="dfb1e0f8b2cc04148fe92a524c78a0265b2a088d73d23e0f3685045e57e15c8c" exitCode=7 Jul 14 17:30:50.565678 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:50.565234 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" event={"ID":"474e162c-9967-4bf0-a876-f073888076c3","Type":"ContainerDied","Data":"dfb1e0f8b2cc04148fe92a524c78a0265b2a088d73d23e0f3685045e57e15c8c"} Jul 14 17:30:50.565678 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:50.565305 2558 scope.go:117] "RemoveContainer" containerID="3d3ca7e925491fe799dcddb5c94809d15531ef71dc44b402ce5e5b7e3f8e22d6" Jul 14 17:30:50.565678 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:50.565625 2558 scope.go:117] "RemoveContainer" containerID="dfb1e0f8b2cc04148fe92a524c78a0265b2a088d73d23e0f3685045e57e15c8c" Jul 14 17:30:50.565870 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:30:50.565834 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2_odh-ai-gateway-infra(474e162c-9967-4bf0-a876-f073888076c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" podUID="474e162c-9967-4bf0-a876-f073888076c3" Jul 14 17:30:51.572502 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:30:51.572473 2558 scope.go:117] "RemoveContainer" containerID="dfb1e0f8b2cc04148fe92a524c78a0265b2a088d73d23e0f3685045e57e15c8c" Jul 14 17:30:51.572923 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:30:51.572734 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2_odh-ai-gateway-infra(474e162c-9967-4bf0-a876-f073888076c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" podUID="474e162c-9967-4bf0-a876-f073888076c3" Jul 14 17:31:06.375932 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:06.375896 2558 scope.go:117] "RemoveContainer" containerID="dfb1e0f8b2cc04148fe92a524c78a0265b2a088d73d23e0f3685045e57e15c8c" Jul 14 17:31:07.660770 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:07.660736 2558 generic.go:356] "Generic (PLEG): container finished" podID="474e162c-9967-4bf0-a876-f073888076c3" containerID="a0d04e3bf214a7d99b440473e295af5b6e377cbd8115d5919e6b1653991da237" exitCode=0 Jul 14 17:31:07.661190 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:07.660810 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" event={"ID":"474e162c-9967-4bf0-a876-f073888076c3","Type":"ContainerDied","Data":"a0d04e3bf214a7d99b440473e295af5b6e377cbd8115d5919e6b1653991da237"} Jul 14 17:31:07.661190 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:07.660877 2558 scope.go:117] "RemoveContainer" containerID="dfb1e0f8b2cc04148fe92a524c78a0265b2a088d73d23e0f3685045e57e15c8c" Jul 14 17:31:08.692225 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:08.692189 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2"] Jul 14 17:31:08.812130 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:08.812104 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" Jul 14 17:31:08.899608 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:08.899536 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/474e162c-9967-4bf0-a876-f073888076c3-maas-api-service-ca\") pod \"474e162c-9967-4bf0-a876-f073888076c3\" (UID: \"474e162c-9967-4bf0-a876-f073888076c3\") " Jul 14 17:31:08.899954 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:08.899930 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474e162c-9967-4bf0-a876-f073888076c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "474e162c-9967-4bf0-a876-f073888076c3" (UID: "474e162c-9967-4bf0-a876-f073888076c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:31:09.001151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:09.001038 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/474e162c-9967-4bf0-a876-f073888076c3-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:31:09.674172 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:09.674085 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" event={"ID":"474e162c-9967-4bf0-a876-f073888076c3","Type":"ContainerDied","Data":"56a372a2899085b350e2e91697dac715244652d80013a670d3f5da6c22d7dd6d"} Jul 14 17:31:09.674172 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:09.674145 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56a372a2899085b350e2e91697dac715244652d80013a670d3f5da6c22d7dd6d" Jul 14 17:31:09.674172 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:09.674104 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2" Jul 14 17:31:09.719037 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:09.718999 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2"] Jul 14 17:31:09.723922 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:09.723895 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-z6lb2"] Jul 14 17:31:11.383849 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:11.383813 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="474e162c-9967-4bf0-a876-f073888076c3" path="/var/lib/kubelet/pods/474e162c-9967-4bf0-a876-f073888076c3/volumes" Jul 14 17:31:48.190413 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.190373 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn"] Jul 14 17:31:48.190946 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.190836 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:31:48.190946 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.190848 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:31:48.190946 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.190865 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:31:48.190946 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.190870 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:31:48.190946 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.190947 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:31:48.191126 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.190959 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:31:48.194222 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.194204 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" Jul 14 17:31:48.196973 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.196955 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:31:48.203039 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.203010 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn"] Jul 14 17:31:48.302258 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.302213 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0f1773-e5bd-4da6-8c18-6e4d17b43934-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn\" (UID: \"2f0f1773-e5bd-4da6-8c18-6e4d17b43934\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" Jul 14 17:31:48.403119 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.403079 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0f1773-e5bd-4da6-8c18-6e4d17b43934-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn\" (UID: \"2f0f1773-e5bd-4da6-8c18-6e4d17b43934\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" Jul 14 17:31:48.403775 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.403753 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0f1773-e5bd-4da6-8c18-6e4d17b43934-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn\" (UID: \"2f0f1773-e5bd-4da6-8c18-6e4d17b43934\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" Jul 14 17:31:48.505713 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.505624 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" Jul 14 17:31:48.651842 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.651807 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn"] Jul 14 17:31:48.654698 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:31:48.654667 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0f1773_e5bd_4da6_8c18_6e4d17b43934.slice/crio-cb1f99c112b6be7974255aa97333b3907fee1d19947bdab1e058240b6a0a5fec WatchSource:0}: Error finding container cb1f99c112b6be7974255aa97333b3907fee1d19947bdab1e058240b6a0a5fec: Status 404 returned error can't find the container with id cb1f99c112b6be7974255aa97333b3907fee1d19947bdab1e058240b6a0a5fec Jul 14 17:31:48.888844 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:48.888813 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" event={"ID":"2f0f1773-e5bd-4da6-8c18-6e4d17b43934","Type":"ContainerStarted","Data":"cb1f99c112b6be7974255aa97333b3907fee1d19947bdab1e058240b6a0a5fec"} Jul 14 17:31:49.895939 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:49.895901 2558 generic.go:356] "Generic (PLEG): container finished" podID="2f0f1773-e5bd-4da6-8c18-6e4d17b43934" containerID="854d6f3777be256ed546a99a5cf3125888af85b0133d2756071e90e947e8c7b6" exitCode=0 Jul 14 17:31:49.896344 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:49.895988 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" event={"ID":"2f0f1773-e5bd-4da6-8c18-6e4d17b43934","Type":"ContainerDied","Data":"854d6f3777be256ed546a99a5cf3125888af85b0133d2756071e90e947e8c7b6"} Jul 14 17:31:51.039420 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:51.039391 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" Jul 14 17:31:51.130218 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:51.130175 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0f1773-e5bd-4da6-8c18-6e4d17b43934-maas-api-service-ca\") pod \"2f0f1773-e5bd-4da6-8c18-6e4d17b43934\" (UID: \"2f0f1773-e5bd-4da6-8c18-6e4d17b43934\") " Jul 14 17:31:51.130586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:51.130532 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f0f1773-e5bd-4da6-8c18-6e4d17b43934-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f0f1773-e5bd-4da6-8c18-6e4d17b43934" (UID: "2f0f1773-e5bd-4da6-8c18-6e4d17b43934"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:31:51.231892 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:51.231797 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f0f1773-e5bd-4da6-8c18-6e4d17b43934-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:31:51.907388 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:51.907348 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" event={"ID":"2f0f1773-e5bd-4da6-8c18-6e4d17b43934","Type":"ContainerDied","Data":"cb1f99c112b6be7974255aa97333b3907fee1d19947bdab1e058240b6a0a5fec"} Jul 14 17:31:51.907626 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:51.907411 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb1f99c112b6be7974255aa97333b3907fee1d19947bdab1e058240b6a0a5fec" Jul 14 17:31:51.907626 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:31:51.907363 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn" Jul 14 17:32:19.766850 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.766809 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv"] Jul 14 17:32:19.767325 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.767285 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:32:19.767325 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.767299 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:32:19.767413 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.767333 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0f1773-e5bd-4da6-8c18-6e4d17b43934" containerName="revoke-keys" Jul 14 17:32:19.767413 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.767338 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0f1773-e5bd-4da6-8c18-6e4d17b43934" containerName="revoke-keys" Jul 14 17:32:19.767476 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.767413 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="474e162c-9967-4bf0-a876-f073888076c3" containerName="revoke-keys" Jul 14 17:32:19.767476 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.767427 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0f1773-e5bd-4da6-8c18-6e4d17b43934" containerName="revoke-keys" Jul 14 17:32:19.770956 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.770932 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" Jul 14 17:32:19.773620 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.773597 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:32:19.779532 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.779503 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv"] Jul 14 17:32:19.827345 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.827306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d056b7c-beb7-4738-807b-72bcc5bce7f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv\" (UID: \"3d056b7c-beb7-4738-807b-72bcc5bce7f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" Jul 14 17:32:19.928886 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.928839 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d056b7c-beb7-4738-807b-72bcc5bce7f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv\" (UID: \"3d056b7c-beb7-4738-807b-72bcc5bce7f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" Jul 14 17:32:19.929482 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:19.929462 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d056b7c-beb7-4738-807b-72bcc5bce7f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv\" (UID: \"3d056b7c-beb7-4738-807b-72bcc5bce7f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" Jul 14 17:32:20.083842 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:20.083748 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" Jul 14 17:32:20.236282 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:20.236250 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv"] Jul 14 17:32:20.238363 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:32:20.238333 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d056b7c_beb7_4738_807b_72bcc5bce7f9.slice/crio-49c35447424845a62fcffde4320f376539671f169616dafe2ce654a54eb617c4 WatchSource:0}: Error finding container 49c35447424845a62fcffde4320f376539671f169616dafe2ce654a54eb617c4: Status 404 returned error can't find the container with id 49c35447424845a62fcffde4320f376539671f169616dafe2ce654a54eb617c4 Jul 14 17:32:21.063557 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:21.063508 2558 generic.go:356] "Generic (PLEG): container finished" podID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerID="32eaaa1a95d99462876aeae35840b1541d457d7b9f06c954304fdb77c5c15e99" exitCode=7 Jul 14 17:32:21.064006 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:21.063579 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" event={"ID":"3d056b7c-beb7-4738-807b-72bcc5bce7f9","Type":"ContainerDied","Data":"32eaaa1a95d99462876aeae35840b1541d457d7b9f06c954304fdb77c5c15e99"} Jul 14 17:32:21.064006 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:21.063645 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" event={"ID":"3d056b7c-beb7-4738-807b-72bcc5bce7f9","Type":"ContainerStarted","Data":"49c35447424845a62fcffde4320f376539671f169616dafe2ce654a54eb617c4"} Jul 14 17:32:21.064006 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:21.063872 2558 scope.go:117] "RemoveContainer" containerID="32eaaa1a95d99462876aeae35840b1541d457d7b9f06c954304fdb77c5c15e99" Jul 14 17:32:22.072733 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:22.072621 2558 generic.go:356] "Generic (PLEG): container finished" podID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerID="eb4310541deb3270275a28802008190186653ac3cfe2634c248f7c1b9da0080f" exitCode=7 Jul 14 17:32:22.072733 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:22.072691 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" event={"ID":"3d056b7c-beb7-4738-807b-72bcc5bce7f9","Type":"ContainerDied","Data":"eb4310541deb3270275a28802008190186653ac3cfe2634c248f7c1b9da0080f"} Jul 14 17:32:22.073250 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:22.072742 2558 scope.go:117] "RemoveContainer" containerID="32eaaa1a95d99462876aeae35840b1541d457d7b9f06c954304fdb77c5c15e99" Jul 14 17:32:22.073250 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:22.072989 2558 scope.go:117] "RemoveContainer" containerID="eb4310541deb3270275a28802008190186653ac3cfe2634c248f7c1b9da0080f" Jul 14 17:32:22.073347 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:32:22.073254 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv_odh-ai-gateway-infra(3d056b7c-beb7-4738-807b-72bcc5bce7f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" Jul 14 17:32:23.079754 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:23.079729 2558 scope.go:117] "RemoveContainer" containerID="eb4310541deb3270275a28802008190186653ac3cfe2634c248f7c1b9da0080f" Jul 14 17:32:23.080140 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:32:23.079949 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv_odh-ai-gateway-infra(3d056b7c-beb7-4738-807b-72bcc5bce7f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" Jul 14 17:32:37.376701 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:37.376665 2558 scope.go:117] "RemoveContainer" containerID="eb4310541deb3270275a28802008190186653ac3cfe2634c248f7c1b9da0080f" Jul 14 17:32:38.167162 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:38.167130 2558 generic.go:356] "Generic (PLEG): container finished" podID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerID="e655bc5624b5f9825b98b0737dcbab3cb78d09ba3cfadd9dee3b66128a7b1ef3" exitCode=0 Jul 14 17:32:38.167356 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:38.167211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" event={"ID":"3d056b7c-beb7-4738-807b-72bcc5bce7f9","Type":"ContainerDied","Data":"e655bc5624b5f9825b98b0737dcbab3cb78d09ba3cfadd9dee3b66128a7b1ef3"} Jul 14 17:32:38.167356 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:38.167285 2558 scope.go:117] "RemoveContainer" containerID="eb4310541deb3270275a28802008190186653ac3cfe2634c248f7c1b9da0080f" Jul 14 17:32:39.210626 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:39.210589 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv"] Jul 14 17:32:39.310218 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:39.310189 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" Jul 14 17:32:39.436789 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:39.436697 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d056b7c-beb7-4738-807b-72bcc5bce7f9-maas-api-service-ca\") pod \"3d056b7c-beb7-4738-807b-72bcc5bce7f9\" (UID: \"3d056b7c-beb7-4738-807b-72bcc5bce7f9\") " Jul 14 17:32:39.437085 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:39.437062 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d056b7c-beb7-4738-807b-72bcc5bce7f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d056b7c-beb7-4738-807b-72bcc5bce7f9" (UID: "3d056b7c-beb7-4738-807b-72bcc5bce7f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:32:39.538088 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:39.538047 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d056b7c-beb7-4738-807b-72bcc5bce7f9-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:32:40.181198 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:40.181165 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" Jul 14 17:32:40.181385 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:40.181211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv" event={"ID":"3d056b7c-beb7-4738-807b-72bcc5bce7f9","Type":"ContainerDied","Data":"49c35447424845a62fcffde4320f376539671f169616dafe2ce654a54eb617c4"} Jul 14 17:32:40.181385 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:40.181320 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49c35447424845a62fcffde4320f376539671f169616dafe2ce654a54eb617c4" Jul 14 17:32:40.220710 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:40.220532 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv"] Jul 14 17:32:40.222940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:40.222920 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-rgrjv"] Jul 14 17:32:41.381845 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:32:41.381813 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" path="/var/lib/kubelet/pods/3d056b7c-beb7-4738-807b-72bcc5bce7f9/volumes" Jul 14 17:33:19.846189 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.846097 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj"] Jul 14 17:33:19.846791 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.846770 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:19.846874 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.846794 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:19.846874 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.846852 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:19.846874 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.846862 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:19.847026 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.846965 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:19.847026 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.846977 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:19.850368 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.850344 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" Jul 14 17:33:19.856919 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.856891 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:33:19.866668 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.866639 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj"] Jul 14 17:33:19.937086 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:19.937044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d2e720-1db6-468e-a76c-6ca39cdd75c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj\" (UID: \"e8d2e720-1db6-468e-a76c-6ca39cdd75c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" Jul 14 17:33:20.037576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:20.037498 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d2e720-1db6-468e-a76c-6ca39cdd75c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj\" (UID: \"e8d2e720-1db6-468e-a76c-6ca39cdd75c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" Jul 14 17:33:20.038126 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:20.038106 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d2e720-1db6-468e-a76c-6ca39cdd75c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj\" (UID: \"e8d2e720-1db6-468e-a76c-6ca39cdd75c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" Jul 14 17:33:20.161082 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:20.160992 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" Jul 14 17:33:20.321718 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:20.321588 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj"] Jul 14 17:33:20.324099 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:33:20.324064 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d2e720_1db6_468e_a76c_6ca39cdd75c3.slice/crio-81ea1883622cbb2998cb22f182cd2e3d31efeb9b8375a985405e42e2fefd5792 WatchSource:0}: Error finding container 81ea1883622cbb2998cb22f182cd2e3d31efeb9b8375a985405e42e2fefd5792: Status 404 returned error can't find the container with id 81ea1883622cbb2998cb22f182cd2e3d31efeb9b8375a985405e42e2fefd5792 Jul 14 17:33:20.398824 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:20.398786 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" event={"ID":"e8d2e720-1db6-468e-a76c-6ca39cdd75c3","Type":"ContainerStarted","Data":"81ea1883622cbb2998cb22f182cd2e3d31efeb9b8375a985405e42e2fefd5792"} Jul 14 17:33:21.405333 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:21.405303 2558 generic.go:356] "Generic (PLEG): container finished" podID="e8d2e720-1db6-468e-a76c-6ca39cdd75c3" containerID="57a771687f57f1241cec6eaf2c9050a9e76855a7157429e9968176c28c804a99" exitCode=0 Jul 14 17:33:21.405755 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:21.405397 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" event={"ID":"e8d2e720-1db6-468e-a76c-6ca39cdd75c3","Type":"ContainerDied","Data":"57a771687f57f1241cec6eaf2c9050a9e76855a7157429e9968176c28c804a99"} Jul 14 17:33:22.557961 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:22.557936 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" Jul 14 17:33:22.664070 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:22.663978 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d2e720-1db6-468e-a76c-6ca39cdd75c3-maas-api-service-ca\") pod \"e8d2e720-1db6-468e-a76c-6ca39cdd75c3\" (UID: \"e8d2e720-1db6-468e-a76c-6ca39cdd75c3\") " Jul 14 17:33:22.664439 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:22.664406 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8d2e720-1db6-468e-a76c-6ca39cdd75c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8d2e720-1db6-468e-a76c-6ca39cdd75c3" (UID: "e8d2e720-1db6-468e-a76c-6ca39cdd75c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:33:22.765085 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:22.765046 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8d2e720-1db6-468e-a76c-6ca39cdd75c3-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:33:23.416992 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:23.416964 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" Jul 14 17:33:23.417176 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:23.416957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj" event={"ID":"e8d2e720-1db6-468e-a76c-6ca39cdd75c3","Type":"ContainerDied","Data":"81ea1883622cbb2998cb22f182cd2e3d31efeb9b8375a985405e42e2fefd5792"} Jul 14 17:33:23.417176 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:23.417088 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81ea1883622cbb2998cb22f182cd2e3d31efeb9b8375a985405e42e2fefd5792" Jul 14 17:33:39.301058 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.300991 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c"] Jul 14 17:33:39.301490 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.301476 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d2e720-1db6-468e-a76c-6ca39cdd75c3" containerName="revoke-keys" Jul 14 17:33:39.301490 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.301492 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d2e720-1db6-468e-a76c-6ca39cdd75c3" containerName="revoke-keys" Jul 14 17:33:39.301615 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.301508 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:39.301615 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.301513 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:39.301615 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.301604 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d2e720-1db6-468e-a76c-6ca39cdd75c3" containerName="revoke-keys" Jul 14 17:33:39.301615 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.301615 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d056b7c-beb7-4738-807b-72bcc5bce7f9" containerName="revoke-keys" Jul 14 17:33:39.304805 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.304788 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" Jul 14 17:33:39.307817 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.307796 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:33:39.314182 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.314154 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c"] Jul 14 17:33:39.435764 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.435729 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da16b20c-cc59-42fe-9a4a-8266a043b478-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30670f0b-s995c\" (UID: \"da16b20c-cc59-42fe-9a4a-8266a043b478\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" Jul 14 17:33:39.536690 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.536653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da16b20c-cc59-42fe-9a4a-8266a043b478-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30670f0b-s995c\" (UID: \"da16b20c-cc59-42fe-9a4a-8266a043b478\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" Jul 14 17:33:39.537287 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.537267 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da16b20c-cc59-42fe-9a4a-8266a043b478-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30670f0b-s995c\" (UID: \"da16b20c-cc59-42fe-9a4a-8266a043b478\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" Jul 14 17:33:39.616222 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.616134 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" Jul 14 17:33:39.757677 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:39.757654 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c"] Jul 14 17:33:39.759565 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:33:39.759509 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda16b20c_cc59_42fe_9a4a_8266a043b478.slice/crio-4090ccd6f53f29d953b44bfd21be33b447aed93329cd749430a8f6ac9763ec84 WatchSource:0}: Error finding container 4090ccd6f53f29d953b44bfd21be33b447aed93329cd749430a8f6ac9763ec84: Status 404 returned error can't find the container with id 4090ccd6f53f29d953b44bfd21be33b447aed93329cd749430a8f6ac9763ec84 Jul 14 17:33:40.508376 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:40.508342 2558 generic.go:356] "Generic (PLEG): container finished" podID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerID="b987c147faeccb77641a158f60cf48735270b9e713355c570fdeedb990f8a4da" exitCode=7 Jul 14 17:33:40.508891 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:40.508428 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" event={"ID":"da16b20c-cc59-42fe-9a4a-8266a043b478","Type":"ContainerDied","Data":"b987c147faeccb77641a158f60cf48735270b9e713355c570fdeedb990f8a4da"} Jul 14 17:33:40.508891 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:40.508484 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" event={"ID":"da16b20c-cc59-42fe-9a4a-8266a043b478","Type":"ContainerStarted","Data":"4090ccd6f53f29d953b44bfd21be33b447aed93329cd749430a8f6ac9763ec84"} Jul 14 17:33:40.508891 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:40.508610 2558 scope.go:117] "RemoveContainer" containerID="b987c147faeccb77641a158f60cf48735270b9e713355c570fdeedb990f8a4da" Jul 14 17:33:41.515792 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:41.515758 2558 generic.go:356] "Generic (PLEG): container finished" podID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerID="e193797085dce18dccea280d1ce75ae57d185b7c07f73bf0c2f820de92b0b6a1" exitCode=7 Jul 14 17:33:41.516290 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:41.515850 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" event={"ID":"da16b20c-cc59-42fe-9a4a-8266a043b478","Type":"ContainerDied","Data":"e193797085dce18dccea280d1ce75ae57d185b7c07f73bf0c2f820de92b0b6a1"} Jul 14 17:33:41.516290 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:41.515915 2558 scope.go:117] "RemoveContainer" containerID="b987c147faeccb77641a158f60cf48735270b9e713355c570fdeedb990f8a4da" Jul 14 17:33:41.516290 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:41.516100 2558 scope.go:117] "RemoveContainer" containerID="e193797085dce18dccea280d1ce75ae57d185b7c07f73bf0c2f820de92b0b6a1" Jul 14 17:33:41.516417 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:33:41.516390 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-30670f0b-s995c_odh-ai-gateway-infra(da16b20c-cc59-42fe-9a4a-8266a043b478)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" Jul 14 17:33:42.525015 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:42.524984 2558 scope.go:117] "RemoveContainer" containerID="e193797085dce18dccea280d1ce75ae57d185b7c07f73bf0c2f820de92b0b6a1" Jul 14 17:33:42.525406 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:33:42.525183 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-30670f0b-s995c_odh-ai-gateway-infra(da16b20c-cc59-42fe-9a4a-8266a043b478)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" Jul 14 17:33:58.376833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:58.376805 2558 scope.go:117] "RemoveContainer" containerID="e193797085dce18dccea280d1ce75ae57d185b7c07f73bf0c2f820de92b0b6a1" Jul 14 17:33:59.617340 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:59.617302 2558 generic.go:356] "Generic (PLEG): container finished" podID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerID="358cc7efee9f788b4628898eceb9cbc69c8a4b4ffc4c48c7dc287ea55bf98e8d" exitCode=0 Jul 14 17:33:59.617777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:59.617379 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" event={"ID":"da16b20c-cc59-42fe-9a4a-8266a043b478","Type":"ContainerDied","Data":"358cc7efee9f788b4628898eceb9cbc69c8a4b4ffc4c48c7dc287ea55bf98e8d"} Jul 14 17:33:59.617777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:33:59.617443 2558 scope.go:117] "RemoveContainer" containerID="e193797085dce18dccea280d1ce75ae57d185b7c07f73bf0c2f820de92b0b6a1" Jul 14 17:34:00.655551 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:00.655492 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c"] Jul 14 17:34:00.767710 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:00.767685 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" Jul 14 17:34:00.857234 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:00.857197 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da16b20c-cc59-42fe-9a4a-8266a043b478-maas-api-service-ca\") pod \"da16b20c-cc59-42fe-9a4a-8266a043b478\" (UID: \"da16b20c-cc59-42fe-9a4a-8266a043b478\") " Jul 14 17:34:00.857648 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:00.857628 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da16b20c-cc59-42fe-9a4a-8266a043b478-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da16b20c-cc59-42fe-9a4a-8266a043b478" (UID: "da16b20c-cc59-42fe-9a4a-8266a043b478"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:00.958884 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:00.958796 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da16b20c-cc59-42fe-9a4a-8266a043b478-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:34:01.630894 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:01.630863 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" Jul 14 17:34:01.631065 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:01.630892 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c" event={"ID":"da16b20c-cc59-42fe-9a4a-8266a043b478","Type":"ContainerDied","Data":"4090ccd6f53f29d953b44bfd21be33b447aed93329cd749430a8f6ac9763ec84"} Jul 14 17:34:01.631065 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:01.630942 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4090ccd6f53f29d953b44bfd21be33b447aed93329cd749430a8f6ac9763ec84" Jul 14 17:34:01.669676 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:01.669573 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c"] Jul 14 17:34:01.679496 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:01.679466 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-s995c"] Jul 14 17:34:03.382461 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:03.382427 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" path="/var/lib/kubelet/pods/da16b20c-cc59-42fe-9a4a-8266a043b478/volumes" Jul 14 17:34:39.382431 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382401 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz"] Jul 14 17:34:39.382926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382815 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.382926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382828 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.382926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382845 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.382926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382851 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.382926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382861 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.382926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382866 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.382926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382923 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.383148 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382933 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.383148 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.382940 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="da16b20c-cc59-42fe-9a4a-8266a043b478" containerName="revoke-keys" Jul 14 17:34:39.386165 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.386149 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" Jul 14 17:34:39.390250 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.390233 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:34:39.396970 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.396944 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz"] Jul 14 17:34:39.436380 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.436338 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d5801f-0115-4a64-8d4d-c8e6c07de98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30670f0b-scqcz\" (UID: \"f7d5801f-0115-4a64-8d4d-c8e6c07de98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" Jul 14 17:34:39.537810 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.537771 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d5801f-0115-4a64-8d4d-c8e6c07de98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30670f0b-scqcz\" (UID: \"f7d5801f-0115-4a64-8d4d-c8e6c07de98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" Jul 14 17:34:39.538437 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.538409 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d5801f-0115-4a64-8d4d-c8e6c07de98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30670f0b-scqcz\" (UID: \"f7d5801f-0115-4a64-8d4d-c8e6c07de98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" Jul 14 17:34:39.698043 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.697946 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" Jul 14 17:34:39.847926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:39.847889 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz"] Jul 14 17:34:39.849101 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:34:39.849077 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d5801f_0115_4a64_8d4d_c8e6c07de98e.slice/crio-17fa6ff0b8e1712f42062e9b32cc5b2ca2cc9e3a82356b7526dd62a019051636 WatchSource:0}: Error finding container 17fa6ff0b8e1712f42062e9b32cc5b2ca2cc9e3a82356b7526dd62a019051636: Status 404 returned error can't find the container with id 17fa6ff0b8e1712f42062e9b32cc5b2ca2cc9e3a82356b7526dd62a019051636 Jul 14 17:34:40.841129 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:40.841041 2558 generic.go:356] "Generic (PLEG): container finished" podID="f7d5801f-0115-4a64-8d4d-c8e6c07de98e" containerID="ee27fa0557475750377c0b3dbc66c8ecfaca3adb0fd623d5fbb7e3fabdf90092" exitCode=0 Jul 14 17:34:40.841495 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:40.841124 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" event={"ID":"f7d5801f-0115-4a64-8d4d-c8e6c07de98e","Type":"ContainerDied","Data":"ee27fa0557475750377c0b3dbc66c8ecfaca3adb0fd623d5fbb7e3fabdf90092"} Jul 14 17:34:40.841495 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:40.841176 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" event={"ID":"f7d5801f-0115-4a64-8d4d-c8e6c07de98e","Type":"ContainerStarted","Data":"17fa6ff0b8e1712f42062e9b32cc5b2ca2cc9e3a82356b7526dd62a019051636"} Jul 14 17:34:41.982276 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:41.982244 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" Jul 14 17:34:42.061273 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:42.061238 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d5801f-0115-4a64-8d4d-c8e6c07de98e-maas-api-service-ca\") pod \"f7d5801f-0115-4a64-8d4d-c8e6c07de98e\" (UID: \"f7d5801f-0115-4a64-8d4d-c8e6c07de98e\") " Jul 14 17:34:42.061628 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:42.061606 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d5801f-0115-4a64-8d4d-c8e6c07de98e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f7d5801f-0115-4a64-8d4d-c8e6c07de98e" (UID: "f7d5801f-0115-4a64-8d4d-c8e6c07de98e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:42.162338 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:42.162228 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7d5801f-0115-4a64-8d4d-c8e6c07de98e-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:34:42.853353 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:42.853316 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" event={"ID":"f7d5801f-0115-4a64-8d4d-c8e6c07de98e","Type":"ContainerDied","Data":"17fa6ff0b8e1712f42062e9b32cc5b2ca2cc9e3a82356b7526dd62a019051636"} Jul 14 17:34:42.853353 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:42.853346 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30670f0b-scqcz" Jul 14 17:34:42.853651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:42.853366 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17fa6ff0b8e1712f42062e9b32cc5b2ca2cc9e3a82356b7526dd62a019051636" Jul 14 17:34:59.197862 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:59.197756 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:34:59.210347 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:59.199355 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:34:59.210347 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:59.199720 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:34:59.210347 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:34:59.201508 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:35:11.686409 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.686369 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5"] Jul 14 17:35:11.686880 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.686798 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7d5801f-0115-4a64-8d4d-c8e6c07de98e" containerName="revoke-keys" Jul 14 17:35:11.686880 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.686810 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d5801f-0115-4a64-8d4d-c8e6c07de98e" containerName="revoke-keys" Jul 14 17:35:11.686971 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.686894 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7d5801f-0115-4a64-8d4d-c8e6c07de98e" containerName="revoke-keys" Jul 14 17:35:11.690298 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.690278 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" Jul 14 17:35:11.693360 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.693336 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:35:11.698866 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.698846 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5"] Jul 14 17:35:11.857774 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.857732 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f84ccc91-f343-4a18-bd19-a50ca45edf7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5\" (UID: \"f84ccc91-f343-4a18-bd19-a50ca45edf7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" Jul 14 17:35:11.959244 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.959148 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f84ccc91-f343-4a18-bd19-a50ca45edf7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5\" (UID: \"f84ccc91-f343-4a18-bd19-a50ca45edf7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" Jul 14 17:35:11.959847 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:11.959826 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f84ccc91-f343-4a18-bd19-a50ca45edf7e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5\" (UID: \"f84ccc91-f343-4a18-bd19-a50ca45edf7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" Jul 14 17:35:12.002590 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:12.002531 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" Jul 14 17:35:12.145877 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:12.145849 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5"] Jul 14 17:35:12.148219 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:35:12.148193 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf84ccc91_f343_4a18_bd19_a50ca45edf7e.slice/crio-bd542f753263e727f89ae6cadaa533323b1b8dca99e0e7a3a78757485a3eef61 WatchSource:0}: Error finding container bd542f753263e727f89ae6cadaa533323b1b8dca99e0e7a3a78757485a3eef61: Status 404 returned error can't find the container with id bd542f753263e727f89ae6cadaa533323b1b8dca99e0e7a3a78757485a3eef61 Jul 14 17:35:13.021305 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:13.021265 2558 generic.go:356] "Generic (PLEG): container finished" podID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerID="2b293d9976c567eba85d2d7f5fdfb1d17506c03121136f91adb4e18562a6e7b8" exitCode=7 Jul 14 17:35:13.021756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:13.021340 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" event={"ID":"f84ccc91-f343-4a18-bd19-a50ca45edf7e","Type":"ContainerDied","Data":"2b293d9976c567eba85d2d7f5fdfb1d17506c03121136f91adb4e18562a6e7b8"} Jul 14 17:35:13.021756 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:13.021380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" event={"ID":"f84ccc91-f343-4a18-bd19-a50ca45edf7e","Type":"ContainerStarted","Data":"bd542f753263e727f89ae6cadaa533323b1b8dca99e0e7a3a78757485a3eef61"} Jul 14 17:35:13.022158 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:13.022136 2558 scope.go:117] "RemoveContainer" containerID="2b293d9976c567eba85d2d7f5fdfb1d17506c03121136f91adb4e18562a6e7b8" Jul 14 17:35:14.029008 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:14.028969 2558 generic.go:356] "Generic (PLEG): container finished" podID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerID="c59b800802f908e7e53d3c00c1f06d4719804b67696f734d30883246a96cdaf7" exitCode=7 Jul 14 17:35:14.029424 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:14.029057 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" event={"ID":"f84ccc91-f343-4a18-bd19-a50ca45edf7e","Type":"ContainerDied","Data":"c59b800802f908e7e53d3c00c1f06d4719804b67696f734d30883246a96cdaf7"} Jul 14 17:35:14.029424 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:14.029116 2558 scope.go:117] "RemoveContainer" containerID="2b293d9976c567eba85d2d7f5fdfb1d17506c03121136f91adb4e18562a6e7b8" Jul 14 17:35:14.029424 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:14.029382 2558 scope.go:117] "RemoveContainer" containerID="c59b800802f908e7e53d3c00c1f06d4719804b67696f734d30883246a96cdaf7" Jul 14 17:35:14.029659 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:35:14.029635 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5_odh-ai-gateway-infra(f84ccc91-f343-4a18-bd19-a50ca45edf7e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" Jul 14 17:35:15.036075 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:15.036044 2558 scope.go:117] "RemoveContainer" containerID="c59b800802f908e7e53d3c00c1f06d4719804b67696f734d30883246a96cdaf7" Jul 14 17:35:15.036470 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:35:15.036318 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5_odh-ai-gateway-infra(f84ccc91-f343-4a18-bd19-a50ca45edf7e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" Jul 14 17:35:29.379123 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:29.379088 2558 scope.go:117] "RemoveContainer" containerID="c59b800802f908e7e53d3c00c1f06d4719804b67696f734d30883246a96cdaf7" Jul 14 17:35:30.127780 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:30.127745 2558 generic.go:356] "Generic (PLEG): container finished" podID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerID="81049a41ce5ae6436cdca7ef663498c8fea78d7fbcc8703bf95eb27435592290" exitCode=0 Jul 14 17:35:30.127921 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:30.127794 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" event={"ID":"f84ccc91-f343-4a18-bd19-a50ca45edf7e","Type":"ContainerDied","Data":"81049a41ce5ae6436cdca7ef663498c8fea78d7fbcc8703bf95eb27435592290"} Jul 14 17:35:30.127921 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:30.127847 2558 scope.go:117] "RemoveContainer" containerID="c59b800802f908e7e53d3c00c1f06d4719804b67696f734d30883246a96cdaf7" Jul 14 17:35:31.174748 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:31.174714 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5"] Jul 14 17:35:31.273143 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:31.273108 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" Jul 14 17:35:31.335775 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:31.335745 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f84ccc91-f343-4a18-bd19-a50ca45edf7e-maas-api-service-ca\") pod \"f84ccc91-f343-4a18-bd19-a50ca45edf7e\" (UID: \"f84ccc91-f343-4a18-bd19-a50ca45edf7e\") " Jul 14 17:35:31.336128 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:31.336105 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f84ccc91-f343-4a18-bd19-a50ca45edf7e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f84ccc91-f343-4a18-bd19-a50ca45edf7e" (UID: "f84ccc91-f343-4a18-bd19-a50ca45edf7e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:35:31.437141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:31.437104 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f84ccc91-f343-4a18-bd19-a50ca45edf7e-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:35:32.142855 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:32.142812 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" event={"ID":"f84ccc91-f343-4a18-bd19-a50ca45edf7e","Type":"ContainerDied","Data":"bd542f753263e727f89ae6cadaa533323b1b8dca99e0e7a3a78757485a3eef61"} Jul 14 17:35:32.143069 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:32.142869 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd542f753263e727f89ae6cadaa533323b1b8dca99e0e7a3a78757485a3eef61" Jul 14 17:35:32.143069 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:32.142833 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5" Jul 14 17:35:32.185784 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:32.185741 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5"] Jul 14 17:35:32.197455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:32.197420 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-qfkp5"] Jul 14 17:35:33.382452 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:35:33.382417 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" path="/var/lib/kubelet/pods/f84ccc91-f343-4a18-bd19-a50ca45edf7e/volumes" Jul 14 17:36:11.749608 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.749568 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn"] Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750003 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750016 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750044 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750052 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750062 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750068 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750139 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.750149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.750148 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:36:11.753436 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.753415 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" Jul 14 17:36:11.756735 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.756711 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:36:11.769929 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.769897 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn"] Jul 14 17:36:11.829291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.829251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33986935-ae55-461a-aca5-825d0329d2a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn\" (UID: \"33986935-ae55-461a-aca5-825d0329d2a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" Jul 14 17:36:11.930715 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.930671 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33986935-ae55-461a-aca5-825d0329d2a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn\" (UID: \"33986935-ae55-461a-aca5-825d0329d2a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" Jul 14 17:36:11.931327 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:11.931306 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33986935-ae55-461a-aca5-825d0329d2a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn\" (UID: \"33986935-ae55-461a-aca5-825d0329d2a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" Jul 14 17:36:12.064400 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:12.064362 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" Jul 14 17:36:12.220131 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:12.220105 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn"] Jul 14 17:36:12.221915 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:36:12.221884 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33986935_ae55_461a_aca5_825d0329d2a2.slice/crio-28c481125a48871e780ce4b5f9ce3fb88b3f019e7881f01e6c9030aa5fb5855e WatchSource:0}: Error finding container 28c481125a48871e780ce4b5f9ce3fb88b3f019e7881f01e6c9030aa5fb5855e: Status 404 returned error can't find the container with id 28c481125a48871e780ce4b5f9ce3fb88b3f019e7881f01e6c9030aa5fb5855e Jul 14 17:36:12.361947 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:12.361849 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" event={"ID":"33986935-ae55-461a-aca5-825d0329d2a2","Type":"ContainerStarted","Data":"28c481125a48871e780ce4b5f9ce3fb88b3f019e7881f01e6c9030aa5fb5855e"} Jul 14 17:36:13.376083 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:13.376047 2558 generic.go:356] "Generic (PLEG): container finished" podID="33986935-ae55-461a-aca5-825d0329d2a2" containerID="93432f4a3b6a9faaf0841bbef069bc5a29b698c8c26ae0d0b045c80341ccb702" exitCode=0 Jul 14 17:36:13.381186 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:13.381158 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" event={"ID":"33986935-ae55-461a-aca5-825d0329d2a2","Type":"ContainerDied","Data":"93432f4a3b6a9faaf0841bbef069bc5a29b698c8c26ae0d0b045c80341ccb702"} Jul 14 17:36:14.518689 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:14.518664 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" Jul 14 17:36:14.656869 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:14.656768 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33986935-ae55-461a-aca5-825d0329d2a2-maas-api-service-ca\") pod \"33986935-ae55-461a-aca5-825d0329d2a2\" (UID: \"33986935-ae55-461a-aca5-825d0329d2a2\") " Jul 14 17:36:14.657144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:14.657119 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33986935-ae55-461a-aca5-825d0329d2a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33986935-ae55-461a-aca5-825d0329d2a2" (UID: "33986935-ae55-461a-aca5-825d0329d2a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:36:14.757917 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:14.757876 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33986935-ae55-461a-aca5-825d0329d2a2-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:36:15.387599 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:15.387569 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" Jul 14 17:36:15.387787 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:15.387631 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn" event={"ID":"33986935-ae55-461a-aca5-825d0329d2a2","Type":"ContainerDied","Data":"28c481125a48871e780ce4b5f9ce3fb88b3f019e7881f01e6c9030aa5fb5855e"} Jul 14 17:36:15.387787 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:36:15.387685 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28c481125a48871e780ce4b5f9ce3fb88b3f019e7881f01e6c9030aa5fb5855e" Jul 14 17:37:59.245962 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:37:59.245923 2558 scope.go:117] "RemoveContainer" containerID="a0d04e3bf214a7d99b440473e295af5b6e377cbd8115d5919e6b1653991da237" Jul 14 17:38:38.309930 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.309891 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh"] Jul 14 17:38:38.310455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.310321 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33986935-ae55-461a-aca5-825d0329d2a2" containerName="revoke-keys" Jul 14 17:38:38.310455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.310332 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="33986935-ae55-461a-aca5-825d0329d2a2" containerName="revoke-keys" Jul 14 17:38:38.310455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.310407 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f84ccc91-f343-4a18-bd19-a50ca45edf7e" containerName="revoke-keys" Jul 14 17:38:38.310455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.310419 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="33986935-ae55-461a-aca5-825d0329d2a2" containerName="revoke-keys" Jul 14 17:38:38.313581 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.313563 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" Jul 14 17:38:38.316698 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.316677 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:38:38.324294 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.324270 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh"] Jul 14 17:38:38.454438 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.454382 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5303b009-f9af-4be9-b162-9c58f44cc1fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-763702f3-pnszh\" (UID: \"5303b009-f9af-4be9-b162-9c58f44cc1fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" Jul 14 17:38:38.555230 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.555161 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5303b009-f9af-4be9-b162-9c58f44cc1fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-763702f3-pnszh\" (UID: \"5303b009-f9af-4be9-b162-9c58f44cc1fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" Jul 14 17:38:38.555819 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.555799 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5303b009-f9af-4be9-b162-9c58f44cc1fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-763702f3-pnszh\" (UID: \"5303b009-f9af-4be9-b162-9c58f44cc1fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" Jul 14 17:38:38.624382 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.624271 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" Jul 14 17:38:38.769823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:38.769766 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh"] Jul 14 17:38:38.772127 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:38:38.772084 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5303b009_f9af_4be9_b162_9c58f44cc1fb.slice/crio-4342ed027f5c760f718d1b8d823924955c1e0d9fd3d9b051f5286ad7476096e8 WatchSource:0}: Error finding container 4342ed027f5c760f718d1b8d823924955c1e0d9fd3d9b051f5286ad7476096e8: Status 404 returned error can't find the container with id 4342ed027f5c760f718d1b8d823924955c1e0d9fd3d9b051f5286ad7476096e8 Jul 14 17:38:39.182890 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:39.182849 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" event={"ID":"5303b009-f9af-4be9-b162-9c58f44cc1fb","Type":"ContainerStarted","Data":"141e31986ae15c344908fdc6e0ef9f954b269fca23a1fb16bc8856ea7beec6d5"} Jul 14 17:38:39.183093 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:39.182904 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" event={"ID":"5303b009-f9af-4be9-b162-9c58f44cc1fb","Type":"ContainerStarted","Data":"4342ed027f5c760f718d1b8d823924955c1e0d9fd3d9b051f5286ad7476096e8"} Jul 14 17:38:39.211229 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:39.211169 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" podStartSLOduration=1.21115056 podStartE2EDuration="1.21115056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:38:39.209963013 +0000 UTC m=+2021.779373783" watchObservedRunningTime="2026-07-14 17:38:39.21115056 +0000 UTC m=+2021.780561372" Jul 14 17:38:40.190405 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:40.190368 2558 generic.go:356] "Generic (PLEG): container finished" podID="5303b009-f9af-4be9-b162-9c58f44cc1fb" containerID="141e31986ae15c344908fdc6e0ef9f954b269fca23a1fb16bc8856ea7beec6d5" exitCode=0 Jul 14 17:38:40.190882 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:40.190456 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" event={"ID":"5303b009-f9af-4be9-b162-9c58f44cc1fb","Type":"ContainerDied","Data":"141e31986ae15c344908fdc6e0ef9f954b269fca23a1fb16bc8856ea7beec6d5"} Jul 14 17:38:41.335653 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:41.335623 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" Jul 14 17:38:41.379900 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:41.379865 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5303b009-f9af-4be9-b162-9c58f44cc1fb-maas-api-service-ca\") pod \"5303b009-f9af-4be9-b162-9c58f44cc1fb\" (UID: \"5303b009-f9af-4be9-b162-9c58f44cc1fb\") " Jul 14 17:38:41.380291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:41.380267 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5303b009-f9af-4be9-b162-9c58f44cc1fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5303b009-f9af-4be9-b162-9c58f44cc1fb" (UID: "5303b009-f9af-4be9-b162-9c58f44cc1fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:38:41.481521 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:41.481427 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5303b009-f9af-4be9-b162-9c58f44cc1fb-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:38:42.202662 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:42.202625 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" event={"ID":"5303b009-f9af-4be9-b162-9c58f44cc1fb","Type":"ContainerDied","Data":"4342ed027f5c760f718d1b8d823924955c1e0d9fd3d9b051f5286ad7476096e8"} Jul 14 17:38:42.202820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:42.202665 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-763702f3-pnszh" Jul 14 17:38:42.202820 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:42.202700 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4342ed027f5c760f718d1b8d823924955c1e0d9fd3d9b051f5286ad7476096e8" Jul 14 17:38:59.266559 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:38:59.266520 2558 scope.go:117] "RemoveContainer" containerID="e655bc5624b5f9825b98b0737dcbab3cb78d09ba3cfadd9dee3b66128a7b1ef3" Jul 14 17:39:59.233822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:39:59.233707 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:39:59.238192 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:39:59.235831 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:39:59.238192 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:39:59.236889 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:39:59.239068 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:39:59.239050 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:39:59.292346 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:39:59.292308 2558 scope.go:117] "RemoveContainer" containerID="358cc7efee9f788b4628898eceb9cbc69c8a4b4ffc4c48c7dc287ea55bf98e8d" Jul 14 17:40:10.219172 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.219134 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq"] Jul 14 17:40:10.219734 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.219578 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5303b009-f9af-4be9-b162-9c58f44cc1fb" containerName="revoke-keys" Jul 14 17:40:10.219734 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.219590 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5303b009-f9af-4be9-b162-9c58f44cc1fb" containerName="revoke-keys" Jul 14 17:40:10.219734 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.219673 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5303b009-f9af-4be9-b162-9c58f44cc1fb" containerName="revoke-keys" Jul 14 17:40:10.222815 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.222797 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" Jul 14 17:40:10.228247 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.228224 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:40:10.238463 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.238435 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq"] Jul 14 17:40:10.274074 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.274037 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841bd5f8-5670-4487-9733-09c82359b78d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq\" (UID: \"841bd5f8-5670-4487-9733-09c82359b78d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" Jul 14 17:40:10.374640 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.374604 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841bd5f8-5670-4487-9733-09c82359b78d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq\" (UID: \"841bd5f8-5670-4487-9733-09c82359b78d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" Jul 14 17:40:10.375319 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.375297 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841bd5f8-5670-4487-9733-09c82359b78d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq\" (UID: \"841bd5f8-5670-4487-9733-09c82359b78d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" Jul 14 17:40:10.534077 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.534035 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" Jul 14 17:40:10.691646 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.691621 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq"] Jul 14 17:40:10.693881 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:40:10.693842 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841bd5f8_5670_4487_9733_09c82359b78d.slice/crio-efce2c7b3d07a43f46004e1094ac1f90a0dc5d7b5555fb2c21be1cf233f28b19 WatchSource:0}: Error finding container efce2c7b3d07a43f46004e1094ac1f90a0dc5d7b5555fb2c21be1cf233f28b19: Status 404 returned error can't find the container with id efce2c7b3d07a43f46004e1094ac1f90a0dc5d7b5555fb2c21be1cf233f28b19 Jul 14 17:40:10.703594 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:10.703564 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" event={"ID":"841bd5f8-5670-4487-9733-09c82359b78d","Type":"ContainerStarted","Data":"efce2c7b3d07a43f46004e1094ac1f90a0dc5d7b5555fb2c21be1cf233f28b19"} Jul 14 17:40:11.710373 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:11.710282 2558 generic.go:356] "Generic (PLEG): container finished" podID="841bd5f8-5670-4487-9733-09c82359b78d" containerID="75c21b07bbfa2691783865db49faf9849acf8d435bc347cf7aa38bd517caf212" exitCode=0 Jul 14 17:40:11.710752 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:11.710364 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" event={"ID":"841bd5f8-5670-4487-9733-09c82359b78d","Type":"ContainerDied","Data":"75c21b07bbfa2691783865db49faf9849acf8d435bc347cf7aa38bd517caf212"} Jul 14 17:40:12.853421 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:12.853388 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" Jul 14 17:40:13.001945 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:13.001838 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841bd5f8-5670-4487-9733-09c82359b78d-maas-api-service-ca\") pod \"841bd5f8-5670-4487-9733-09c82359b78d\" (UID: \"841bd5f8-5670-4487-9733-09c82359b78d\") " Jul 14 17:40:13.002286 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:13.002259 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/841bd5f8-5670-4487-9733-09c82359b78d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "841bd5f8-5670-4487-9733-09c82359b78d" (UID: "841bd5f8-5670-4487-9733-09c82359b78d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:40:13.103360 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:13.103314 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/841bd5f8-5670-4487-9733-09c82359b78d-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:40:13.723094 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:13.723007 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" Jul 14 17:40:13.723094 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:13.723049 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq" event={"ID":"841bd5f8-5670-4487-9733-09c82359b78d","Type":"ContainerDied","Data":"efce2c7b3d07a43f46004e1094ac1f90a0dc5d7b5555fb2c21be1cf233f28b19"} Jul 14 17:40:13.723290 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:40:13.723096 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efce2c7b3d07a43f46004e1094ac1f90a0dc5d7b5555fb2c21be1cf233f28b19" Jul 14 17:41:59.330208 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:41:59.330171 2558 scope.go:117] "RemoveContainer" containerID="81049a41ce5ae6436cdca7ef663498c8fea78d7fbcc8703bf95eb27435592290" Jul 14 17:42:25.826050 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.825964 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz"] Jul 14 17:42:25.826592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.826440 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="841bd5f8-5670-4487-9733-09c82359b78d" containerName="revoke-keys" Jul 14 17:42:25.826592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.826454 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="841bd5f8-5670-4487-9733-09c82359b78d" containerName="revoke-keys" Jul 14 17:42:25.826592 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.826535 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="841bd5f8-5670-4487-9733-09c82359b78d" containerName="revoke-keys" Jul 14 17:42:25.829924 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.829899 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:25.833227 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.833203 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5zbzj\"" Jul 14 17:42:25.834170 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.834147 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-1b06a1-serving-cert\"" Jul 14 17:42:25.856949 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.856910 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz"] Jul 14 17:42:25.937734 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.937694 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv5jk\" (UniqueName: \"kubernetes.io/projected/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-kube-api-access-gv5jk\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:25.937734 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.937734 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:25.937953 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:25.937861 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.038776 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.038733 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5jk\" (UniqueName: \"kubernetes.io/projected/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-kube-api-access-gv5jk\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.038776 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.038777 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.039060 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.038843 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.039060 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:42:26.038989 2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-serving-cert: secret "maas-api-e2e-inf-b-1b06a1-serving-cert" not found Jul 14 17:42:26.039060 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:42:26.039057 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls podName:1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3 nodeName:}" failed. No retries permitted until 2026-07-14 17:42:26.539033876 +0000 UTC m=+2249.108444630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls") pod "maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" (UID: "1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3") : secret "maas-api-e2e-inf-b-1b06a1-serving-cert" not found Jul 14 17:42:26.039743 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.039715 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.052144 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.052112 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv5jk\" (UniqueName: \"kubernetes.io/projected/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-kube-api-access-gv5jk\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.543728 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.543689 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.546152 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.546131 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls\") pod \"maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.743008 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.742971 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:26.915791 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.915764 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz"] Jul 14 17:42:26.918312 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:42:26.918287 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a5c5b81_8658_4851_9e9c_6a8b50e1e1c3.slice/crio-7fe92486e7b783a010fc67f76686efa90abf6fb57468e18714f62e9fd7e1c08b WatchSource:0}: Error finding container 7fe92486e7b783a010fc67f76686efa90abf6fb57468e18714f62e9fd7e1c08b: Status 404 returned error can't find the container with id 7fe92486e7b783a010fc67f76686efa90abf6fb57468e18714f62e9fd7e1c08b Jul 14 17:42:26.920141 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:26.920123 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:42:27.454802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:27.454767 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" event={"ID":"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3","Type":"ContainerStarted","Data":"7fe92486e7b783a010fc67f76686efa90abf6fb57468e18714f62e9fd7e1c08b"} Jul 14 17:42:29.469149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:29.469094 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" event={"ID":"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3","Type":"ContainerStarted","Data":"3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4"} Jul 14 17:42:29.469605 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:29.469199 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:42:29.494777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:29.494717 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" podStartSLOduration=2.137728092 podStartE2EDuration="4.494697599s" totalImagesPullingTime="2.356969507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:42:29.491260264 +0000 UTC m=+2252.060671064" watchObservedRunningTime="2026-07-14 17:42:29.494697599 +0000 UTC m=+2252.064108370" Jul 14 17:42:35.482531 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:42:35.482496 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:43:14.235798 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.235761 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb"] Jul 14 17:43:14.244022 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.243887 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" Jul 14 17:43:14.247064 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.247031 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:43:14.249030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.248991 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb"] Jul 14 17:43:14.313354 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.313315 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26476a42-6f5b-43d2-837c-f3c860e342f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb\" (UID: \"26476a42-6f5b-43d2-837c-f3c860e342f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" Jul 14 17:43:14.414384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.414345 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26476a42-6f5b-43d2-837c-f3c860e342f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb\" (UID: \"26476a42-6f5b-43d2-837c-f3c860e342f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" Jul 14 17:43:14.415003 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.414981 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26476a42-6f5b-43d2-837c-f3c860e342f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb\" (UID: \"26476a42-6f5b-43d2-837c-f3c860e342f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" Jul 14 17:43:14.557369 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.557328 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" Jul 14 17:43:14.702576 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.702528 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb"] Jul 14 17:43:14.705038 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:43:14.705010 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26476a42_6f5b_43d2_837c_f3c860e342f8.slice/crio-0906a3474c25c9c4d0a6321612ba0eb2a5bb96aefa04674ba2bd5bec3c54834e WatchSource:0}: Error finding container 0906a3474c25c9c4d0a6321612ba0eb2a5bb96aefa04674ba2bd5bec3c54834e: Status 404 returned error can't find the container with id 0906a3474c25c9c4d0a6321612ba0eb2a5bb96aefa04674ba2bd5bec3c54834e Jul 14 17:43:14.720933 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:14.720898 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" event={"ID":"26476a42-6f5b-43d2-837c-f3c860e342f8","Type":"ContainerStarted","Data":"0906a3474c25c9c4d0a6321612ba0eb2a5bb96aefa04674ba2bd5bec3c54834e"} Jul 14 17:43:15.728081 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:15.728049 2558 generic.go:356] "Generic (PLEG): container finished" podID="26476a42-6f5b-43d2-837c-f3c860e342f8" containerID="ab070a64459bae740c94d06b7755b1fad2c191a9d3abbb6213b7b6bde7cac375" exitCode=0 Jul 14 17:43:15.728480 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:15.728140 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" event={"ID":"26476a42-6f5b-43d2-837c-f3c860e342f8","Type":"ContainerDied","Data":"ab070a64459bae740c94d06b7755b1fad2c191a9d3abbb6213b7b6bde7cac375"} Jul 14 17:43:16.872291 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:16.872258 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" Jul 14 17:43:17.040195 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:17.040159 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26476a42-6f5b-43d2-837c-f3c860e342f8-maas-api-service-ca\") pod \"26476a42-6f5b-43d2-837c-f3c860e342f8\" (UID: \"26476a42-6f5b-43d2-837c-f3c860e342f8\") " Jul 14 17:43:17.040582 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:17.040524 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26476a42-6f5b-43d2-837c-f3c860e342f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26476a42-6f5b-43d2-837c-f3c860e342f8" (UID: "26476a42-6f5b-43d2-837c-f3c860e342f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:43:17.141338 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:17.141301 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26476a42-6f5b-43d2-837c-f3c860e342f8-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:43:17.739937 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:17.739854 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" Jul 14 17:43:17.739937 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:17.739890 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb" event={"ID":"26476a42-6f5b-43d2-837c-f3c860e342f8","Type":"ContainerDied","Data":"0906a3474c25c9c4d0a6321612ba0eb2a5bb96aefa04674ba2bd5bec3c54834e"} Jul 14 17:43:17.739937 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:17.739937 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0906a3474c25c9c4d0a6321612ba0eb2a5bb96aefa04674ba2bd5bec3c54834e" Jul 14 17:43:40.426228 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.426182 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r"] Jul 14 17:43:40.426896 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.426874 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26476a42-6f5b-43d2-837c-f3c860e342f8" containerName="revoke-keys" Jul 14 17:43:40.426988 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.426899 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="26476a42-6f5b-43d2-837c-f3c860e342f8" containerName="revoke-keys" Jul 14 17:43:40.427039 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.427007 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="26476a42-6f5b-43d2-837c-f3c860e342f8" containerName="revoke-keys" Jul 14 17:43:40.430383 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.430358 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" Jul 14 17:43:40.433995 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.433970 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:43:40.452194 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.452162 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r"] Jul 14 17:43:40.572993 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.572952 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/463548b2-4a5d-42fb-99df-59de50413a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r\" (UID: \"463548b2-4a5d-42fb-99df-59de50413a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" Jul 14 17:43:40.673661 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.673620 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/463548b2-4a5d-42fb-99df-59de50413a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r\" (UID: \"463548b2-4a5d-42fb-99df-59de50413a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" Jul 14 17:43:40.674265 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.674245 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/463548b2-4a5d-42fb-99df-59de50413a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r\" (UID: \"463548b2-4a5d-42fb-99df-59de50413a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" Jul 14 17:43:40.741384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:40.741287 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" Jul 14 17:43:41.098281 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:41.098251 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r"] Jul 14 17:43:41.099832 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:43:41.099800 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod463548b2_4a5d_42fb_99df_59de50413a55.slice/crio-4257beb744d9eb3e21163562415cb7c4dfd689a2c97ba5aca8f971288c620238 WatchSource:0}: Error finding container 4257beb744d9eb3e21163562415cb7c4dfd689a2c97ba5aca8f971288c620238: Status 404 returned error can't find the container with id 4257beb744d9eb3e21163562415cb7c4dfd689a2c97ba5aca8f971288c620238 Jul 14 17:43:41.877963 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:41.877919 2558 generic.go:356] "Generic (PLEG): container finished" podID="463548b2-4a5d-42fb-99df-59de50413a55" containerID="a570616f1d408c5922c2c11c5de7b40d7709ae60d6340fa9899d171a6dd596d1" exitCode=0 Jul 14 17:43:41.878341 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:41.877961 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" event={"ID":"463548b2-4a5d-42fb-99df-59de50413a55","Type":"ContainerDied","Data":"a570616f1d408c5922c2c11c5de7b40d7709ae60d6340fa9899d171a6dd596d1"} Jul 14 17:43:41.878341 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:41.878022 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" event={"ID":"463548b2-4a5d-42fb-99df-59de50413a55","Type":"ContainerStarted","Data":"4257beb744d9eb3e21163562415cb7c4dfd689a2c97ba5aca8f971288c620238"} Jul 14 17:43:43.019851 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:43.019820 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" Jul 14 17:43:43.098283 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:43.098252 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/463548b2-4a5d-42fb-99df-59de50413a55-maas-api-service-ca\") pod \"463548b2-4a5d-42fb-99df-59de50413a55\" (UID: \"463548b2-4a5d-42fb-99df-59de50413a55\") " Jul 14 17:43:43.098627 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:43.098604 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/463548b2-4a5d-42fb-99df-59de50413a55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "463548b2-4a5d-42fb-99df-59de50413a55" (UID: "463548b2-4a5d-42fb-99df-59de50413a55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:43:43.199712 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:43.199621 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/463548b2-4a5d-42fb-99df-59de50413a55-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:43:43.891236 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:43.891199 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" event={"ID":"463548b2-4a5d-42fb-99df-59de50413a55","Type":"ContainerDied","Data":"4257beb744d9eb3e21163562415cb7c4dfd689a2c97ba5aca8f971288c620238"} Jul 14 17:43:43.891453 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:43.891259 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4257beb744d9eb3e21163562415cb7c4dfd689a2c97ba5aca8f971288c620238" Jul 14 17:43:43.891453 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:43.891213 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r" Jul 14 17:43:51.100709 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.100623 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz"] Jul 14 17:43:51.101133 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.100836 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" podUID="1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" containerName="maas-api" containerID="cri-o://3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4" gracePeriod=30 Jul 14 17:43:51.351295 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.351205 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:43:51.481725 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.481682 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls\") pod \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " Jul 14 17:43:51.481927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.481776 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-openshift-service-ca\") pod \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " Jul 14 17:43:51.481927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.481869 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv5jk\" (UniqueName: \"kubernetes.io/projected/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-kube-api-access-gv5jk\") pod \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\" (UID: \"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3\") " Jul 14 17:43:51.482161 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.482130 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" (UID: "1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:43:51.484081 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.484057 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" (UID: "1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:43:51.484164 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.484094 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-kube-api-access-gv5jk" (OuterVolumeSpecName: "kube-api-access-gv5jk") pod "1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" (UID: "1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3"). InnerVolumeSpecName "kube-api-access-gv5jk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:43:51.583495 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.583456 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv5jk\" (UniqueName: \"kubernetes.io/projected/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-kube-api-access-gv5jk\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:43:51.583495 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.583487 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-maas-api-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:43:51.583495 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.583496 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3-openshift-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:43:51.933712 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.933678 2558 generic.go:356] "Generic (PLEG): container finished" podID="1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" containerID="3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4" exitCode=0 Jul 14 17:43:51.933898 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.933724 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" Jul 14 17:43:51.933898 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.933744 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" event={"ID":"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3","Type":"ContainerDied","Data":"3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4"} Jul 14 17:43:51.933898 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.933786 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz" event={"ID":"1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3","Type":"ContainerDied","Data":"7fe92486e7b783a010fc67f76686efa90abf6fb57468e18714f62e9fd7e1c08b"} Jul 14 17:43:51.933898 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.933808 2558 scope.go:117] "RemoveContainer" containerID="3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4" Jul 14 17:43:51.943572 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.943535 2558 scope.go:117] "RemoveContainer" containerID="3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4" Jul 14 17:43:51.943849 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:43:51.943833 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4\": container with ID starting with 3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4 not found: ID does not exist" containerID="3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4" Jul 14 17:43:51.943897 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.943857 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4"} err="failed to get container status \"3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4\": rpc error: code = NotFound desc = could not find container \"3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4\": container with ID starting with 3409c831c854764c37d39f7611886d0c9ad216decb0cf41cdc1a6b3a43efafd4 not found: ID does not exist" Jul 14 17:43:51.960926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.960898 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz"] Jul 14 17:43:51.966941 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:51.966914 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1b06a1-74855f45d9-9xzfz"] Jul 14 17:43:53.382669 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:43:53.382627 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" path="/var/lib/kubelet/pods/1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3/volumes" Jul 14 17:44:18.261696 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.261656 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt"] Jul 14 17:44:18.262432 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.262408 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" containerName="maas-api" Jul 14 17:44:18.262517 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.262437 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" containerName="maas-api" Jul 14 17:44:18.262517 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.262451 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="463548b2-4a5d-42fb-99df-59de50413a55" containerName="revoke-keys" Jul 14 17:44:18.262517 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.262460 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="463548b2-4a5d-42fb-99df-59de50413a55" containerName="revoke-keys" Jul 14 17:44:18.262698 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.262574 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a5c5b81-8658-4851-9e9c-6a8b50e1e1c3" containerName="maas-api" Jul 14 17:44:18.262698 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.262599 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="463548b2-4a5d-42fb-99df-59de50413a55" containerName="revoke-keys" Jul 14 17:44:18.266571 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.266527 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.269981 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.269956 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5zbzj\"" Jul 14 17:44:18.270132 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.269964 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-6f8249-serving-cert\"" Jul 14 17:44:18.283953 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.283925 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt"] Jul 14 17:44:18.336617 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.336576 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbxsj\" (UniqueName: \"kubernetes.io/projected/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-kube-api-access-tbxsj\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.336822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.336650 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.336822 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.336689 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-openshift-service-ca\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.438002 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.437964 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbxsj\" (UniqueName: \"kubernetes.io/projected/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-kube-api-access-tbxsj\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.438211 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.438034 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.438211 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.438089 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-openshift-service-ca\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.438211 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:44:18.438154 2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-serving-cert: secret "maas-api-e2e-api-b-6f8249-serving-cert" not found Jul 14 17:44:18.438415 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:44:18.438246 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls podName:71ffe93c-4549-4f86-abc9-b4a5fcce35fb nodeName:}" failed. No retries permitted until 2026-07-14 17:44:18.938224427 +0000 UTC m=+2361.507635178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls") pod "maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" (UID: "71ffe93c-4549-4f86-abc9-b4a5fcce35fb") : secret "maas-api-e2e-api-b-6f8249-serving-cert" not found Jul 14 17:44:18.438723 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.438702 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-openshift-service-ca\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.459224 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.459189 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbxsj\" (UniqueName: \"kubernetes.io/projected/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-kube-api-access-tbxsj\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.943453 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.943394 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:18.945864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:18.945838 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls\") pod \"maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:19.181836 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:19.181794 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:19.360607 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:19.360576 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt"] Jul 14 17:44:19.362177 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:44:19.362148 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ffe93c_4549_4f86_abc9_b4a5fcce35fb.slice/crio-0bfccef8e61b4b80ca06effca917429b8528861026b2384767b0b096c1be2630 WatchSource:0}: Error finding container 0bfccef8e61b4b80ca06effca917429b8528861026b2384767b0b096c1be2630: Status 404 returned error can't find the container with id 0bfccef8e61b4b80ca06effca917429b8528861026b2384767b0b096c1be2630 Jul 14 17:44:20.103351 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:20.103314 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" event={"ID":"71ffe93c-4549-4f86-abc9-b4a5fcce35fb","Type":"ContainerStarted","Data":"e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1"} Jul 14 17:44:20.103586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:20.103383 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" event={"ID":"71ffe93c-4549-4f86-abc9-b4a5fcce35fb","Type":"ContainerStarted","Data":"0bfccef8e61b4b80ca06effca917429b8528861026b2384767b0b096c1be2630"} Jul 14 17:44:20.103586 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:20.103418 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:20.129521 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:20.129406 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" podStartSLOduration=1.629544649 podStartE2EDuration="2.129389083s" totalImagesPullingTime="499.844434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:44:20.126422226 +0000 UTC m=+2362.695832999" watchObservedRunningTime="2026-07-14 17:44:20.129389083 +0000 UTC m=+2362.698799854" Jul 14 17:44:26.114032 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:26.114000 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:44:30.689448 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:30.689402 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b"] Jul 14 17:44:30.693256 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:30.693233 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" Jul 14 17:44:30.696065 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:30.696038 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:44:30.704149 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:30.704110 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b"] Jul 14 17:44:30.859071 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:30.859031 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c6e938-3a82-4fdb-845c-69ed972e9b2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-083b58-gqf2b\" (UID: \"89c6e938-3a82-4fdb-845c-69ed972e9b2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" Jul 14 17:44:30.960710 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:30.960602 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c6e938-3a82-4fdb-845c-69ed972e9b2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-083b58-gqf2b\" (UID: \"89c6e938-3a82-4fdb-845c-69ed972e9b2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" Jul 14 17:44:30.961297 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:30.961269 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c6e938-3a82-4fdb-845c-69ed972e9b2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-083b58-gqf2b\" (UID: \"89c6e938-3a82-4fdb-845c-69ed972e9b2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" Jul 14 17:44:31.004558 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:31.004510 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" Jul 14 17:44:31.156497 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:31.156463 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b"] Jul 14 17:44:31.158227 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:44:31.158193 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c6e938_3a82_4fdb_845c_69ed972e9b2a.slice/crio-94489b6aed8f762e069794aaac4043a47ff5eb4e2a70638bdf9bf2e27c25b03b WatchSource:0}: Error finding container 94489b6aed8f762e069794aaac4043a47ff5eb4e2a70638bdf9bf2e27c25b03b: Status 404 returned error can't find the container with id 94489b6aed8f762e069794aaac4043a47ff5eb4e2a70638bdf9bf2e27c25b03b Jul 14 17:44:32.170072 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:32.170020 2558 generic.go:356] "Generic (PLEG): container finished" podID="89c6e938-3a82-4fdb-845c-69ed972e9b2a" containerID="0298836ae4a4e366c8366e6b93f5009e35ab97a989adacfb18e3411bf0a68ffb" exitCode=0 Jul 14 17:44:32.170689 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:32.170095 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" event={"ID":"89c6e938-3a82-4fdb-845c-69ed972e9b2a","Type":"ContainerDied","Data":"0298836ae4a4e366c8366e6b93f5009e35ab97a989adacfb18e3411bf0a68ffb"} Jul 14 17:44:32.170689 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:32.170156 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" event={"ID":"89c6e938-3a82-4fdb-845c-69ed972e9b2a","Type":"ContainerStarted","Data":"94489b6aed8f762e069794aaac4043a47ff5eb4e2a70638bdf9bf2e27c25b03b"} Jul 14 17:44:33.320657 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:33.320628 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" Jul 14 17:44:33.486651 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:33.486516 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c6e938-3a82-4fdb-845c-69ed972e9b2a-maas-api-service-ca\") pod \"89c6e938-3a82-4fdb-845c-69ed972e9b2a\" (UID: \"89c6e938-3a82-4fdb-845c-69ed972e9b2a\") " Jul 14 17:44:33.486907 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:33.486883 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c6e938-3a82-4fdb-845c-69ed972e9b2a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89c6e938-3a82-4fdb-845c-69ed972e9b2a" (UID: "89c6e938-3a82-4fdb-845c-69ed972e9b2a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:44:33.588148 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:33.588109 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89c6e938-3a82-4fdb-845c-69ed972e9b2a-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:44:34.183856 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:34.183822 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" event={"ID":"89c6e938-3a82-4fdb-845c-69ed972e9b2a","Type":"ContainerDied","Data":"94489b6aed8f762e069794aaac4043a47ff5eb4e2a70638bdf9bf2e27c25b03b"} Jul 14 17:44:34.183856 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:34.183842 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-083b58-gqf2b" Jul 14 17:44:34.184084 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:34.183875 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94489b6aed8f762e069794aaac4043a47ff5eb4e2a70638bdf9bf2e27c25b03b" Jul 14 17:44:56.893655 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:56.893615 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q"] Jul 14 17:44:56.894152 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:56.894072 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c6e938-3a82-4fdb-845c-69ed972e9b2a" containerName="revoke-keys" Jul 14 17:44:56.894152 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:56.894084 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c6e938-3a82-4fdb-845c-69ed972e9b2a" containerName="revoke-keys" Jul 14 17:44:56.894233 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:56.894180 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c6e938-3a82-4fdb-845c-69ed972e9b2a" containerName="revoke-keys" Jul 14 17:44:56.897687 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:56.897668 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" Jul 14 17:44:56.900571 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:56.900534 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:44:56.908210 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:56.908180 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q"] Jul 14 17:44:57.012014 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:57.011969 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0277e5e-4375-4bb3-b357-74a356e2e0a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6f8249-78k6q\" (UID: \"b0277e5e-4375-4bb3-b357-74a356e2e0a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" Jul 14 17:44:57.112852 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:57.112807 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0277e5e-4375-4bb3-b357-74a356e2e0a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6f8249-78k6q\" (UID: \"b0277e5e-4375-4bb3-b357-74a356e2e0a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" Jul 14 17:44:57.113442 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:57.113420 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0277e5e-4375-4bb3-b357-74a356e2e0a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6f8249-78k6q\" (UID: \"b0277e5e-4375-4bb3-b357-74a356e2e0a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" Jul 14 17:44:57.209940 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:57.209852 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" Jul 14 17:44:57.358499 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:57.358461 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q"] Jul 14 17:44:57.360251 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:44:57.360219 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0277e5e_4375_4bb3_b357_74a356e2e0a6.slice/crio-9bba107db6c78e922b02bfe903df8eb447bf348bbbd34123ddf8e91432764de8 WatchSource:0}: Error finding container 9bba107db6c78e922b02bfe903df8eb447bf348bbbd34123ddf8e91432764de8: Status 404 returned error can't find the container with id 9bba107db6c78e922b02bfe903df8eb447bf348bbbd34123ddf8e91432764de8 Jul 14 17:44:58.325030 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:58.324998 2558 generic.go:356] "Generic (PLEG): container finished" podID="b0277e5e-4375-4bb3-b357-74a356e2e0a6" containerID="10897af244f6aebbca7da36011efb663e0281780cb4fe677b3a241c3549282fb" exitCode=0 Jul 14 17:44:58.325490 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:58.325051 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" event={"ID":"b0277e5e-4375-4bb3-b357-74a356e2e0a6","Type":"ContainerDied","Data":"10897af244f6aebbca7da36011efb663e0281780cb4fe677b3a241c3549282fb"} Jul 14 17:44:58.325490 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:58.325091 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" event={"ID":"b0277e5e-4375-4bb3-b357-74a356e2e0a6","Type":"ContainerStarted","Data":"9bba107db6c78e922b02bfe903df8eb447bf348bbbd34123ddf8e91432764de8"} Jul 14 17:44:59.272668 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.272627 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:44:59.275235 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.275211 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:44:59.276626 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.276606 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:44:59.279070 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.279049 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:44:59.467110 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.467084 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" Jul 14 17:44:59.534855 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.534760 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0277e5e-4375-4bb3-b357-74a356e2e0a6-maas-api-service-ca\") pod \"b0277e5e-4375-4bb3-b357-74a356e2e0a6\" (UID: \"b0277e5e-4375-4bb3-b357-74a356e2e0a6\") " Jul 14 17:44:59.535186 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.535160 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0277e5e-4375-4bb3-b357-74a356e2e0a6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0277e5e-4375-4bb3-b357-74a356e2e0a6" (UID: "b0277e5e-4375-4bb3-b357-74a356e2e0a6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:44:59.636230 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:44:59.636178 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0277e5e-4375-4bb3-b357-74a356e2e0a6-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:45:00.171486 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.171444 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp"] Jul 14 17:45:00.172156 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.172138 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0277e5e-4375-4bb3-b357-74a356e2e0a6" containerName="revoke-keys" Jul 14 17:45:00.172207 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.172161 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0277e5e-4375-4bb3-b357-74a356e2e0a6" containerName="revoke-keys" Jul 14 17:45:00.172290 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.172279 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0277e5e-4375-4bb3-b357-74a356e2e0a6" containerName="revoke-keys" Jul 14 17:45:00.175977 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.175953 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4"] Jul 14 17:45:00.176107 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.176083 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" Jul 14 17:45:00.179475 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.179454 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" Jul 14 17:45:00.187175 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.185659 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp"] Jul 14 17:45:00.189787 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.189751 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4"] Jul 14 17:45:00.192777 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.192752 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" Jul 14 17:45:00.200835 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.200811 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" Jul 14 17:45:00.339271 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.339232 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" Jul 14 17:45:00.339456 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.339220 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6f8249-78k6q" event={"ID":"b0277e5e-4375-4bb3-b357-74a356e2e0a6","Type":"ContainerDied","Data":"9bba107db6c78e922b02bfe903df8eb447bf348bbbd34123ddf8e91432764de8"} Jul 14 17:45:00.339456 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.339332 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bba107db6c78e922b02bfe903df8eb447bf348bbbd34123ddf8e91432764de8" Jul 14 17:45:00.362835 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.362796 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp"] Jul 14 17:45:00.367086 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:45:00.367060 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7679756a_af0e_4c37_8fee_f3a154f72d15.slice/crio-1c440a0a7b49f9ddda09e06b2b330f3820c93cda2c0cc393440da988168e8949 WatchSource:0}: Error finding container 1c440a0a7b49f9ddda09e06b2b330f3820c93cda2c0cc393440da988168e8949: Status 404 returned error can't find the container with id 1c440a0a7b49f9ddda09e06b2b330f3820c93cda2c0cc393440da988168e8949 Jul 14 17:45:00.603489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:00.603456 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4"] Jul 14 17:45:00.604293 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:45:00.604266 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fd30c0_1fe0_42c8_80c9_5826b4465340.slice/crio-c0a008a15513ee501986b1602c28be55c2c9baa9beacecfd6f5b02e5255662cd WatchSource:0}: Error finding container c0a008a15513ee501986b1602c28be55c2c9baa9beacecfd6f5b02e5255662cd: Status 404 returned error can't find the container with id c0a008a15513ee501986b1602c28be55c2c9baa9beacecfd6f5b02e5255662cd Jul 14 17:45:01.347080 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:01.347041 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" event={"ID":"7679756a-af0e-4c37-8fee-f3a154f72d15","Type":"ContainerStarted","Data":"4fbd8be3557e55bf6a8507926a0299f33b95ea217e3546c4e019be3257039a89"} Jul 14 17:45:01.347284 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:01.347100 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" event={"ID":"7679756a-af0e-4c37-8fee-f3a154f72d15","Type":"ContainerStarted","Data":"1c440a0a7b49f9ddda09e06b2b330f3820c93cda2c0cc393440da988168e8949"} Jul 14 17:45:01.348662 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:01.348635 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" event={"ID":"10fd30c0-1fe0-42c8-80c9-5826b4465340","Type":"ContainerStarted","Data":"0054d4a42f2654460edbdae68231113de983ed83dcb2e20c0d66e873086bc3df"} Jul 14 17:45:01.348788 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:01.348678 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" event={"ID":"10fd30c0-1fe0-42c8-80c9-5826b4465340","Type":"ContainerStarted","Data":"c0a008a15513ee501986b1602c28be55c2c9baa9beacecfd6f5b02e5255662cd"} Jul 14 17:45:01.385225 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:01.385169 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" podStartSLOduration=1.385152384 podStartE2EDuration="1.385152384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:45:01.366252384 +0000 UTC m=+2403.935663158" watchObservedRunningTime="2026-07-14 17:45:01.385152384 +0000 UTC m=+2403.954563155" Jul 14 17:45:01.385426 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:01.385250 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" podStartSLOduration=1.385244523 podStartE2EDuration="1.385244523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:45:01.384386079 +0000 UTC m=+2403.953796850" watchObservedRunningTime="2026-07-14 17:45:01.385244523 +0000 UTC m=+2403.954655293" Jul 14 17:45:02.356722 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:02.356684 2558 generic.go:356] "Generic (PLEG): container finished" podID="7679756a-af0e-4c37-8fee-f3a154f72d15" containerID="4fbd8be3557e55bf6a8507926a0299f33b95ea217e3546c4e019be3257039a89" exitCode=0 Jul 14 17:45:02.357228 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:02.356779 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" event={"ID":"7679756a-af0e-4c37-8fee-f3a154f72d15","Type":"ContainerDied","Data":"4fbd8be3557e55bf6a8507926a0299f33b95ea217e3546c4e019be3257039a89"} Jul 14 17:45:02.358361 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:02.358335 2558 generic.go:356] "Generic (PLEG): container finished" podID="10fd30c0-1fe0-42c8-80c9-5826b4465340" containerID="0054d4a42f2654460edbdae68231113de983ed83dcb2e20c0d66e873086bc3df" exitCode=0 Jul 14 17:45:02.358482 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:02.358420 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" event={"ID":"10fd30c0-1fe0-42c8-80c9-5826b4465340","Type":"ContainerDied","Data":"0054d4a42f2654460edbdae68231113de983ed83dcb2e20c0d66e873086bc3df"} Jul 14 17:45:03.540715 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:03.540688 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" Jul 14 17:45:03.562328 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:03.562292 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" Jul 14 17:45:04.378183 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:04.378155 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" Jul 14 17:45:04.378380 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:04.378204 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4" event={"ID":"10fd30c0-1fe0-42c8-80c9-5826b4465340","Type":"ContainerDied","Data":"c0a008a15513ee501986b1602c28be55c2c9baa9beacecfd6f5b02e5255662cd"} Jul 14 17:45:04.378380 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:04.378250 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0a008a15513ee501986b1602c28be55c2c9baa9beacecfd6f5b02e5255662cd" Jul 14 17:45:04.379493 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:04.379465 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" event={"ID":"7679756a-af0e-4c37-8fee-f3a154f72d15","Type":"ContainerDied","Data":"1c440a0a7b49f9ddda09e06b2b330f3820c93cda2c0cc393440da988168e8949"} Jul 14 17:45:04.379493 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:04.379490 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734185-drlrp" Jul 14 17:45:04.379665 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:04.379512 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c440a0a7b49f9ddda09e06b2b330f3820c93cda2c0cc393440da988168e8949" Jul 14 17:45:07.463960 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.463923 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt"] Jul 14 17:45:07.464395 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.464146 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" podUID="71ffe93c-4549-4f86-abc9-b4a5fcce35fb" containerName="maas-api" containerID="cri-o://e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1" gracePeriod=30 Jul 14 17:45:07.504065 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.504020 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4"] Jul 14 17:45:07.506632 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.506604 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-6f8249-29734185-4t9z4"] Jul 14 17:45:07.730316 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.730233 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:45:07.823910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.823857 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-openshift-service-ca\") pod \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " Jul 14 17:45:07.824099 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.823920 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbxsj\" (UniqueName: \"kubernetes.io/projected/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-kube-api-access-tbxsj\") pod \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " Jul 14 17:45:07.824099 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.824038 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls\") pod \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\" (UID: \"71ffe93c-4549-4f86-abc9-b4a5fcce35fb\") " Jul 14 17:45:07.824254 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.824225 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71ffe93c-4549-4f86-abc9-b4a5fcce35fb" (UID: "71ffe93c-4549-4f86-abc9-b4a5fcce35fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:45:07.824384 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.824369 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-openshift-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:45:07.826079 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.826056 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71ffe93c-4549-4f86-abc9-b4a5fcce35fb" (UID: "71ffe93c-4549-4f86-abc9-b4a5fcce35fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:45:07.826194 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.826175 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-kube-api-access-tbxsj" (OuterVolumeSpecName: "kube-api-access-tbxsj") pod "71ffe93c-4549-4f86-abc9-b4a5fcce35fb" (UID: "71ffe93c-4549-4f86-abc9-b4a5fcce35fb"). InnerVolumeSpecName "kube-api-access-tbxsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:45:07.925507 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.925465 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbxsj\" (UniqueName: \"kubernetes.io/projected/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-kube-api-access-tbxsj\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:45:07.925507 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:07.925498 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ffe93c-4549-4f86-abc9-b4a5fcce35fb-maas-api-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:45:08.404489 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.404443 2558 generic.go:356] "Generic (PLEG): container finished" podID="71ffe93c-4549-4f86-abc9-b4a5fcce35fb" containerID="e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1" exitCode=0 Jul 14 17:45:08.404732 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.404511 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" Jul 14 17:45:08.404732 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.404558 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" event={"ID":"71ffe93c-4549-4f86-abc9-b4a5fcce35fb","Type":"ContainerDied","Data":"e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1"} Jul 14 17:45:08.404732 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.404635 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt" event={"ID":"71ffe93c-4549-4f86-abc9-b4a5fcce35fb","Type":"ContainerDied","Data":"0bfccef8e61b4b80ca06effca917429b8528861026b2384767b0b096c1be2630"} Jul 14 17:45:08.404732 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.404668 2558 scope.go:117] "RemoveContainer" containerID="e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1" Jul 14 17:45:08.415463 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.415438 2558 scope.go:117] "RemoveContainer" containerID="e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1" Jul 14 17:45:08.415784 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:45:08.415763 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1\": container with ID starting with e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1 not found: ID does not exist" containerID="e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1" Jul 14 17:45:08.415833 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.415796 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1"} err="failed to get container status \"e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1\": rpc error: code = NotFound desc = could not find container \"e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1\": container with ID starting with e2d20106aa7f41328f5c2483d8ba4977774e0282a2704555d45271d0db5c10a1 not found: ID does not exist" Jul 14 17:45:08.428295 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.428260 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt"] Jul 14 17:45:08.434757 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:08.434728 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-6f8249-d4f9c5cc8-2j7kt"] Jul 14 17:45:09.385094 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:09.385057 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10fd30c0-1fe0-42c8-80c9-5826b4465340" path="/var/lib/kubelet/pods/10fd30c0-1fe0-42c8-80c9-5826b4465340/volumes" Jul 14 17:45:09.385596 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:09.385430 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71ffe93c-4549-4f86-abc9-b4a5fcce35fb" path="/var/lib/kubelet/pods/71ffe93c-4549-4f86-abc9-b4a5fcce35fb/volumes" Jul 14 17:45:34.593737 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.593698 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn"] Jul 14 17:45:34.594263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594200 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7679756a-af0e-4c37-8fee-f3a154f72d15" containerName="cleanup" Jul 14 17:45:34.594263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594215 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7679756a-af0e-4c37-8fee-f3a154f72d15" containerName="cleanup" Jul 14 17:45:34.594263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594228 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71ffe93c-4549-4f86-abc9-b4a5fcce35fb" containerName="maas-api" Jul 14 17:45:34.594263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594235 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="71ffe93c-4549-4f86-abc9-b4a5fcce35fb" containerName="maas-api" Jul 14 17:45:34.594263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594253 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10fd30c0-1fe0-42c8-80c9-5826b4465340" containerName="cleanup" Jul 14 17:45:34.594263 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594259 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="10fd30c0-1fe0-42c8-80c9-5826b4465340" containerName="cleanup" Jul 14 17:45:34.594478 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594329 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="10fd30c0-1fe0-42c8-80c9-5826b4465340" containerName="cleanup" Jul 14 17:45:34.594478 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594339 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7679756a-af0e-4c37-8fee-f3a154f72d15" containerName="cleanup" Jul 14 17:45:34.594478 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.594349 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="71ffe93c-4549-4f86-abc9-b4a5fcce35fb" containerName="maas-api" Jul 14 17:45:34.597773 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.597749 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.600758 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.600729 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5zbzj\"" Jul 14 17:45:34.600926 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.600730 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-df3dc5-serving-cert\"" Jul 14 17:45:34.610976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.610944 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn"] Jul 14 17:45:34.691530 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.691481 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.691775 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.691588 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/419a7613-11d9-4eb7-ad30-b8ed279d61ed-kube-api-access-8pn6f\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.691843 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.691765 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/419a7613-11d9-4eb7-ad30-b8ed279d61ed-openshift-service-ca\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.793027 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.792987 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/419a7613-11d9-4eb7-ad30-b8ed279d61ed-openshift-service-ca\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.793238 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.793060 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.793238 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.793128 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/419a7613-11d9-4eb7-ad30-b8ed279d61ed-kube-api-access-8pn6f\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.793444 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:45:34.793391 2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-serving-cert: secret "maas-api-e2e-shared-b-df3dc5-serving-cert" not found Jul 14 17:45:34.793571 ip-10-0-128-240 kubenswrapper[2558]: E0714 17:45:34.793472 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls podName:419a7613-11d9-4eb7-ad30-b8ed279d61ed nodeName:}" failed. No retries permitted until 2026-07-14 17:45:35.293446705 +0000 UTC m=+2437.862857455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls") pod "maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" (UID: "419a7613-11d9-4eb7-ad30-b8ed279d61ed") : secret "maas-api-e2e-shared-b-df3dc5-serving-cert" not found Jul 14 17:45:34.793802 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.793780 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/419a7613-11d9-4eb7-ad30-b8ed279d61ed-openshift-service-ca\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:34.802226 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:34.802198 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/419a7613-11d9-4eb7-ad30-b8ed279d61ed-kube-api-access-8pn6f\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:35.297605 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:35.297526 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:35.300130 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:35.300097 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls\") pod \"maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:35.509459 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:35.509414 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:35.659013 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:35.658980 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn"] Jul 14 17:45:35.660969 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:45:35.660931 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419a7613_11d9_4eb7_ad30_b8ed279d61ed.slice/crio-17b6b2773f125e76cb13fbfca77413ba39f75aed60959004a6e3be80c40efa54 WatchSource:0}: Error finding container 17b6b2773f125e76cb13fbfca77413ba39f75aed60959004a6e3be80c40efa54: Status 404 returned error can't find the container with id 17b6b2773f125e76cb13fbfca77413ba39f75aed60959004a6e3be80c40efa54 Jul 14 17:45:36.574193 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:36.574156 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" event={"ID":"419a7613-11d9-4eb7-ad30-b8ed279d61ed","Type":"ContainerStarted","Data":"600866ff1b86536f3678c74beacbd10ea88ba8327db25eed85b3bfbcb6f3662e"} Jul 14 17:45:36.574416 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:36.574211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" event={"ID":"419a7613-11d9-4eb7-ad30-b8ed279d61ed","Type":"ContainerStarted","Data":"17b6b2773f125e76cb13fbfca77413ba39f75aed60959004a6e3be80c40efa54"} Jul 14 17:45:36.574416 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:36.574240 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:45:36.594455 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:36.594398 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" podStartSLOduration=2.193626201 podStartE2EDuration="2.59437457s" totalImagesPullingTime="400.748369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:45:36.593118632 +0000 UTC m=+2439.162529451" watchObservedRunningTime="2026-07-14 17:45:36.59437457 +0000 UTC m=+2439.163785342" Jul 14 17:45:42.586812 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:45:42.586779 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:49:59.311895 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:49:59.311776 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:49:59.314207 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:49:59.314186 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:49:59.317772 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:49:59.317746 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:49:59.319942 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:49:59.319915 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6sjrd_3ca33e99-b10d-406f-b4fa-3be9bcbd7467/ovn-acl-logging/0.log" Jul 14 17:51:43.770203 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:43.770166 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg"] Jul 14 17:51:43.773976 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:43.773957 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" Jul 14 17:51:43.776751 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:43.776727 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:51:43.782416 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:43.782388 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg"] Jul 14 17:51:43.868615 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:43.868580 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8145aaf9-929c-4ec1-ac4f-092f93b65f9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg\" (UID: \"8145aaf9-929c-4ec1-ac4f-092f93b65f9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" Jul 14 17:51:43.969884 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:43.969830 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8145aaf9-929c-4ec1-ac4f-092f93b65f9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg\" (UID: \"8145aaf9-929c-4ec1-ac4f-092f93b65f9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" Jul 14 17:51:43.970497 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:43.970475 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8145aaf9-929c-4ec1-ac4f-092f93b65f9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg\" (UID: \"8145aaf9-929c-4ec1-ac4f-092f93b65f9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" Jul 14 17:51:44.086416 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:44.086331 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" Jul 14 17:51:44.233156 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:44.233126 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg"] Jul 14 17:51:44.234602 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:51:44.234571 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8145aaf9_929c_4ec1_ac4f_092f93b65f9d.slice/crio-afbaecd92518c47608be56b010949ac4bc49a0b134eaa936d3a201a01ae2eadf WatchSource:0}: Error finding container afbaecd92518c47608be56b010949ac4bc49a0b134eaa936d3a201a01ae2eadf: Status 404 returned error can't find the container with id afbaecd92518c47608be56b010949ac4bc49a0b134eaa936d3a201a01ae2eadf Jul 14 17:51:44.739969 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:44.739866 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" event={"ID":"8145aaf9-929c-4ec1-ac4f-092f93b65f9d","Type":"ContainerStarted","Data":"4507965ded44aac290dedcef60f2c74c72bf511d4309eacec01ad98147b1bea2"} Jul 14 17:51:44.739969 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:44.739932 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" event={"ID":"8145aaf9-929c-4ec1-ac4f-092f93b65f9d","Type":"ContainerStarted","Data":"afbaecd92518c47608be56b010949ac4bc49a0b134eaa936d3a201a01ae2eadf"} Jul 14 17:51:44.780718 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:44.780657 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" podStartSLOduration=1.780637942 podStartE2EDuration="1.780637942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:51:44.780432934 +0000 UTC m=+2807.349843704" watchObservedRunningTime="2026-07-14 17:51:44.780637942 +0000 UTC m=+2807.350048713" Jul 14 17:51:45.747302 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:45.747265 2558 generic.go:356] "Generic (PLEG): container finished" podID="8145aaf9-929c-4ec1-ac4f-092f93b65f9d" containerID="4507965ded44aac290dedcef60f2c74c72bf511d4309eacec01ad98147b1bea2" exitCode=0 Jul 14 17:51:45.747506 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:45.747347 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" event={"ID":"8145aaf9-929c-4ec1-ac4f-092f93b65f9d","Type":"ContainerDied","Data":"4507965ded44aac290dedcef60f2c74c72bf511d4309eacec01ad98147b1bea2"} Jul 14 17:51:46.893298 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:46.893268 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" Jul 14 17:51:46.997130 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:46.997090 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8145aaf9-929c-4ec1-ac4f-092f93b65f9d-maas-api-service-ca\") pod \"8145aaf9-929c-4ec1-ac4f-092f93b65f9d\" (UID: \"8145aaf9-929c-4ec1-ac4f-092f93b65f9d\") " Jul 14 17:51:46.997505 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:46.997485 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8145aaf9-929c-4ec1-ac4f-092f93b65f9d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8145aaf9-929c-4ec1-ac4f-092f93b65f9d" (UID: "8145aaf9-929c-4ec1-ac4f-092f93b65f9d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:51:47.098681 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:47.098640 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8145aaf9-929c-4ec1-ac4f-092f93b65f9d-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:51:47.760920 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:47.760835 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" Jul 14 17:51:47.760920 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:47.760849 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg" event={"ID":"8145aaf9-929c-4ec1-ac4f-092f93b65f9d","Type":"ContainerDied","Data":"afbaecd92518c47608be56b010949ac4bc49a0b134eaa936d3a201a01ae2eadf"} Jul 14 17:51:47.760920 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:47.760900 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afbaecd92518c47608be56b010949ac4bc49a0b134eaa936d3a201a01ae2eadf" Jul 14 17:51:59.455487 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:51:59.455452 2558 scope.go:117] "RemoveContainer" containerID="0054d4a42f2654460edbdae68231113de983ed83dcb2e20c0d66e873086bc3df" Jul 14 17:52:10.100684 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.100646 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x"] Jul 14 17:52:10.101210 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.101149 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8145aaf9-929c-4ec1-ac4f-092f93b65f9d" containerName="revoke-keys" Jul 14 17:52:10.101210 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.101164 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8145aaf9-929c-4ec1-ac4f-092f93b65f9d" containerName="revoke-keys" Jul 14 17:52:10.101298 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.101239 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8145aaf9-929c-4ec1-ac4f-092f93b65f9d" containerName="revoke-keys" Jul 14 17:52:10.104705 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.104688 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" Jul 14 17:52:10.109751 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.109726 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hf29\"" Jul 14 17:52:10.116151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.116120 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x"] Jul 14 17:52:10.229811 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.229766 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2189d-6b77-4f93-82b1-265811097ca5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x\" (UID: \"0ad2189d-6b77-4f93-82b1-265811097ca5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" Jul 14 17:52:10.331230 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.331196 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2189d-6b77-4f93-82b1-265811097ca5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x\" (UID: \"0ad2189d-6b77-4f93-82b1-265811097ca5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" Jul 14 17:52:10.331899 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.331879 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2189d-6b77-4f93-82b1-265811097ca5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x\" (UID: \"0ad2189d-6b77-4f93-82b1-265811097ca5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" Jul 14 17:52:10.417057 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.416958 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" Jul 14 17:52:10.591766 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.591736 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x"] Jul 14 17:52:10.593797 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:52:10.593759 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad2189d_6b77_4f93_82b1_265811097ca5.slice/crio-338b82edfafb212317c251c6162dfa031c4cd9e2aa4911bf88f59e87d06835fd WatchSource:0}: Error finding container 338b82edfafb212317c251c6162dfa031c4cd9e2aa4911bf88f59e87d06835fd: Status 404 returned error can't find the container with id 338b82edfafb212317c251c6162dfa031c4cd9e2aa4911bf88f59e87d06835fd Jul 14 17:52:10.892509 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.892474 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" event={"ID":"0ad2189d-6b77-4f93-82b1-265811097ca5","Type":"ContainerStarted","Data":"338b82edfafb212317c251c6162dfa031c4cd9e2aa4911bf88f59e87d06835fd"} Jul 14 17:52:10.921603 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:10.921519 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" podStartSLOduration=0.921501783 podStartE2EDuration="921.501783ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:52:10.919881819 +0000 UTC m=+2833.489292591" watchObservedRunningTime="2026-07-14 17:52:10.921501783 +0000 UTC m=+2833.490912552" Jul 14 17:52:11.906511 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:11.906466 2558 generic.go:356] "Generic (PLEG): container finished" podID="0ad2189d-6b77-4f93-82b1-265811097ca5" containerID="6ff182da1cc8ab9743edc0f817b86e111209441185133e1314e43397b5069750" exitCode=0 Jul 14 17:52:11.906928 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:11.906571 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" event={"ID":"0ad2189d-6b77-4f93-82b1-265811097ca5","Type":"ContainerDied","Data":"6ff182da1cc8ab9743edc0f817b86e111209441185133e1314e43397b5069750"} Jul 14 17:52:13.055475 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:13.055444 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" Jul 14 17:52:13.160529 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:13.160477 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2189d-6b77-4f93-82b1-265811097ca5-maas-api-service-ca\") pod \"0ad2189d-6b77-4f93-82b1-265811097ca5\" (UID: \"0ad2189d-6b77-4f93-82b1-265811097ca5\") " Jul 14 17:52:13.160868 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:13.160842 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad2189d-6b77-4f93-82b1-265811097ca5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ad2189d-6b77-4f93-82b1-265811097ca5" (UID: "0ad2189d-6b77-4f93-82b1-265811097ca5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:52:13.160970 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:13.160957 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2189d-6b77-4f93-82b1-265811097ca5-maas-api-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:52:13.919953 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:13.919925 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" Jul 14 17:52:13.920165 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:13.919912 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x" event={"ID":"0ad2189d-6b77-4f93-82b1-265811097ca5","Type":"ContainerDied","Data":"338b82edfafb212317c251c6162dfa031c4cd9e2aa4911bf88f59e87d06835fd"} Jul 14 17:52:13.920165 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:13.920066 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="338b82edfafb212317c251c6162dfa031c4cd9e2aa4911bf88f59e87d06835fd" Jul 14 17:52:20.745680 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:20.745641 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn"] Jul 14 17:52:20.746209 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:20.745907 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" podUID="419a7613-11d9-4eb7-ad30-b8ed279d61ed" containerName="maas-api" containerID="cri-o://600866ff1b86536f3678c74beacbd10ea88ba8327db25eed85b3bfbcb6f3662e" gracePeriod=30 Jul 14 17:52:20.963063 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:20.963031 2558 generic.go:356] "Generic (PLEG): container finished" podID="419a7613-11d9-4eb7-ad30-b8ed279d61ed" containerID="600866ff1b86536f3678c74beacbd10ea88ba8327db25eed85b3bfbcb6f3662e" exitCode=0 Jul 14 17:52:20.963255 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:20.963107 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" event={"ID":"419a7613-11d9-4eb7-ad30-b8ed279d61ed","Type":"ContainerDied","Data":"600866ff1b86536f3678c74beacbd10ea88ba8327db25eed85b3bfbcb6f3662e"} Jul 14 17:52:20.999151 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:20.999086 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:52:21.034961 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.034930 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls\") pod \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " Jul 14 17:52:21.035158 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.034988 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/419a7613-11d9-4eb7-ad30-b8ed279d61ed-openshift-service-ca\") pod \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " Jul 14 17:52:21.035158 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.035068 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/419a7613-11d9-4eb7-ad30-b8ed279d61ed-kube-api-access-8pn6f\") pod \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\" (UID: \"419a7613-11d9-4eb7-ad30-b8ed279d61ed\") " Jul 14 17:52:21.035429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.035405 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/419a7613-11d9-4eb7-ad30-b8ed279d61ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "419a7613-11d9-4eb7-ad30-b8ed279d61ed" (UID: "419a7613-11d9-4eb7-ad30-b8ed279d61ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:52:21.037121 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.037100 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "419a7613-11d9-4eb7-ad30-b8ed279d61ed" (UID: "419a7613-11d9-4eb7-ad30-b8ed279d61ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:52:21.037247 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.037229 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/419a7613-11d9-4eb7-ad30-b8ed279d61ed-kube-api-access-8pn6f" (OuterVolumeSpecName: "kube-api-access-8pn6f") pod "419a7613-11d9-4eb7-ad30-b8ed279d61ed" (UID: "419a7613-11d9-4eb7-ad30-b8ed279d61ed"). InnerVolumeSpecName "kube-api-access-8pn6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:52:21.135997 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.135948 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/419a7613-11d9-4eb7-ad30-b8ed279d61ed-maas-api-tls\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:52:21.135997 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.135989 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/419a7613-11d9-4eb7-ad30-b8ed279d61ed-openshift-service-ca\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:52:21.135997 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.136003 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pn6f\" (UniqueName: \"kubernetes.io/projected/419a7613-11d9-4eb7-ad30-b8ed279d61ed-kube-api-access-8pn6f\") on node \"ip-10-0-128-240.ec2.internal\" DevicePath \"\"" Jul 14 17:52:21.970924 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.970891 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" Jul 14 17:52:21.971419 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.970924 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn" event={"ID":"419a7613-11d9-4eb7-ad30-b8ed279d61ed","Type":"ContainerDied","Data":"17b6b2773f125e76cb13fbfca77413ba39f75aed60959004a6e3be80c40efa54"} Jul 14 17:52:21.971419 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.970988 2558 scope.go:117] "RemoveContainer" containerID="600866ff1b86536f3678c74beacbd10ea88ba8327db25eed85b3bfbcb6f3662e" Jul 14 17:52:21.992617 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.992580 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn"] Jul 14 17:52:21.996910 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:21.996882 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-df3dc5-797c7dc6b7-fq5xn"] Jul 14 17:52:23.381906 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:23.381870 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="419a7613-11d9-4eb7-ad30-b8ed279d61ed" path="/var/lib/kubelet/pods/419a7613-11d9-4eb7-ad30-b8ed279d61ed/volumes" Jul 14 17:52:42.633648 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:42.633614 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-xpk77_4b907c15-81f1-454a-b936-6d5ae0d886e5/copy-manifests/0.log" Jul 14 17:52:42.642138 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:42.642106 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-xpk77_4b907c15-81f1-454a-b936-6d5ae0d886e5/manager/0.log" Jul 14 17:52:43.246947 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:43.246918 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734170-dw6tx_1f94cabf-8f25-40c9-8206-fac685af861d/cleanup/0.log" Jul 14 17:52:43.358064 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:43.358041 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734185-drlrp_7679756a-af0e-4c37-8fee-f3a154f72d15/cleanup/0.log" Jul 14 17:52:43.474993 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:43.474961 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-30670f0b-scqcz_f7d5801f-0115-4a64-8d4d-c8e6c07de98e/revoke-keys/0.log" Jul 14 17:52:43.587935 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:43.587906 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-fa58286c-gxfjn_2f0f1773-e5bd-4da6-8c18-6e4d17b43934/revoke-keys/0.log" Jul 14 17:52:43.699441 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:43.699406 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-083b58-gqf2b_89c6e938-3a82-4fdb-845c-69ed972e9b2a/revoke-keys/0.log" Jul 14 17:52:43.814406 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:43.814374 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-6f8249-78k6q_b0277e5e-4375-4bb3-b357-74a356e2e0a6/revoke-keys/0.log" Jul 14 17:52:43.932370 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:43.932288 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-74fcaf1a-gtfbn_33986935-ae55-461a-aca5-825d0329d2a2/revoke-keys/0.log" Jul 14 17:52:44.068280 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.068250 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-d44b45-sdfqb_26476a42-6f5b-43d2-837c-f3c860e342f8/revoke-keys/0.log" Jul 14 17:52:44.181764 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.181734 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-1b06a1-fcz5r_463548b2-4a5d-42fb-99df-59de50413a55/revoke-keys/0.log" Jul 14 17:52:44.294719 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.294687 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-288a19fd-d9kfj_e8d2e720-1db6-468e-a76c-6ca39cdd75c3/revoke-keys/0.log" Jul 14 17:52:44.407498 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.407467 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-763702f3-pnszh_5303b009-f9af-4be9-b162-9c58f44cc1fb/revoke-keys/0.log" Jul 14 17:52:44.524411 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.524377 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-9cc7492b-tbpjq_841bd5f8-5670-4487-9733-09c82359b78d/revoke-keys/0.log" Jul 14 17:52:44.631965 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.631875 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-246d80-cp7vg_8145aaf9-929c-4ec1-ac4f-092f93b65f9d/revoke-keys/0.log" Jul 14 17:52:44.740114 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.740079 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-df3dc5-prq8x_0ad2189d-6b77-4f93-82b1-265811097ca5/revoke-keys/0.log" Jul 14 17:52:44.848441 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:44.848409 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-z47b9_f8883ec1-f2a6-4982-b254-1b1be580d9c5/postgres/0.log" Jul 14 17:52:46.817522 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:46.817486 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7x6p5_16670199-c56b-49bf-94c9-62ead84b2332/manager/0.log" Jul 14 17:52:47.543177 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:47.543134 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vsq7s_3920de5e-8ecc-4746-b3e5-edb2d9d3d688/discovery/0.log" Jul 14 17:52:48.281929 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:48.281897 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7497bbf74b-2lkjk_fef52ed7-0972-4f2e-8cc0-2780b7f59855/router/0.log" Jul 14 17:52:48.854622 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:48.854595 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jlprl_65659adf-b41f-44f7-a472-35898be98e61/main/0.log" Jul 14 17:52:56.287480 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:56.287444 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s29kb_ba8203bc-44c2-4091-ae5d-f0fb93aba210/global-pull-secret-syncer/0.log" Jul 14 17:52:56.442506 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:56.442472 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wckws_555eafcf-ccb1-4454-b367-f5b038305022/konnectivity-agent/0.log" Jul 14 17:52:56.517180 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:52:56.517149 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-240.ec2.internal_d778c27593e0f865e91ef9b3785245bd/haproxy/0.log" Jul 14 17:53:02.422631 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:02.422524 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7x6p5_16670199-c56b-49bf-94c9-62ead84b2332/manager/0.log" Jul 14 17:53:04.186779 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.186686 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3afc7dab-5f74-4b95-9690-a9af36743454/alertmanager/0.log" Jul 14 17:53:04.221664 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.221629 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3afc7dab-5f74-4b95-9690-a9af36743454/config-reloader/0.log" Jul 14 17:53:04.261821 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.261781 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3afc7dab-5f74-4b95-9690-a9af36743454/kube-rbac-proxy-web/0.log" Jul 14 17:53:04.287493 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.287466 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3afc7dab-5f74-4b95-9690-a9af36743454/kube-rbac-proxy/0.log" Jul 14 17:53:04.316254 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.316225 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3afc7dab-5f74-4b95-9690-a9af36743454/kube-rbac-proxy-metric/0.log" Jul 14 17:53:04.353444 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.353419 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3afc7dab-5f74-4b95-9690-a9af36743454/prom-label-proxy/0.log" Jul 14 17:53:04.389714 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.389687 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3afc7dab-5f74-4b95-9690-a9af36743454/init-config-reloader/0.log" Jul 14 17:53:04.464823 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.464742 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-b4wp2_5c3c9438-260b-4e36-91c9-d6d8b300965d/cluster-monitoring-operator/0.log" Jul 14 17:53:04.499835 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.499803 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-rqqlg_d947656e-0a97-44b0-9218-5f36dbc3d5cb/kube-state-metrics/0.log" Jul 14 17:53:04.530429 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.530394 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-rqqlg_d947656e-0a97-44b0-9218-5f36dbc3d5cb/kube-rbac-proxy-main/0.log" Jul 14 17:53:04.558013 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.557981 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-rqqlg_d947656e-0a97-44b0-9218-5f36dbc3d5cb/kube-rbac-proxy-self/0.log" Jul 14 17:53:04.624184 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.624155 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-v8jhw_df56e700-da50-4f28-8718-6529646955ec/monitoring-plugin/0.log" Jul 14 17:53:04.834575 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.834527 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8vj5_e6ddef09-af7f-4493-a6dc-809f0b4e7190/node-exporter/0.log" Jul 14 17:53:04.859467 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.859432 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8vj5_e6ddef09-af7f-4493-a6dc-809f0b4e7190/kube-rbac-proxy/0.log" Jul 14 17:53:04.883457 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.883429 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w8vj5_e6ddef09-af7f-4493-a6dc-809f0b4e7190/init-textfile/0.log" Jul 14 17:53:04.913879 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.913852 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-t4hpc_fe093afe-a132-497b-815b-229484d06d6a/kube-rbac-proxy-main/0.log" Jul 14 17:53:04.940690 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.940665 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-t4hpc_fe093afe-a132-497b-815b-229484d06d6a/kube-rbac-proxy-self/0.log" Jul 14 17:53:04.966331 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:04.966301 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-t4hpc_fe093afe-a132-497b-815b-229484d06d6a/openshift-state-metrics/0.log" Jul 14 17:53:05.000218 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.000189 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a5e27b3-26bf-4c65-9f93-70647dcff7f8/prometheus/0.log" Jul 14 17:53:05.028650 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.028616 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a5e27b3-26bf-4c65-9f93-70647dcff7f8/config-reloader/0.log" Jul 14 17:53:05.055193 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.055167 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a5e27b3-26bf-4c65-9f93-70647dcff7f8/thanos-sidecar/0.log" Jul 14 17:53:05.079178 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.079142 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a5e27b3-26bf-4c65-9f93-70647dcff7f8/kube-rbac-proxy-web/0.log" Jul 14 17:53:05.104224 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.104144 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a5e27b3-26bf-4c65-9f93-70647dcff7f8/kube-rbac-proxy/0.log" Jul 14 17:53:05.134244 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.134215 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a5e27b3-26bf-4c65-9f93-70647dcff7f8/kube-rbac-proxy-thanos/0.log" Jul 14 17:53:05.159936 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.159909 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3a5e27b3-26bf-4c65-9f93-70647dcff7f8/init-config-reloader/0.log" Jul 14 17:53:05.245332 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.245301 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-bb9ls_7d22368a-6361-4381-9a34-ca1aca90a464/prometheus-operator-admission-webhook/0.log" Jul 14 17:53:05.275911 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.275880 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-746b495b48-kfwnf_ab2a64e7-c20b-4fcd-9bd4-4775679845fc/telemeter-client/0.log" Jul 14 17:53:05.302863 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.302825 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-746b495b48-kfwnf_ab2a64e7-c20b-4fcd-9bd4-4775679845fc/reload/0.log" Jul 14 17:53:05.325871 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.325837 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-746b495b48-kfwnf_ab2a64e7-c20b-4fcd-9bd4-4775679845fc/kube-rbac-proxy/0.log" Jul 14 17:53:05.364861 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.364778 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf6fdfc65-rr5pc_fc9d977a-1414-4de2-97eb-cf9368d51887/thanos-query/0.log" Jul 14 17:53:05.389125 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.389100 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf6fdfc65-rr5pc_fc9d977a-1414-4de2-97eb-cf9368d51887/kube-rbac-proxy-web/0.log" Jul 14 17:53:05.413841 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.413812 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf6fdfc65-rr5pc_fc9d977a-1414-4de2-97eb-cf9368d51887/kube-rbac-proxy/0.log" Jul 14 17:53:05.438160 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.438127 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf6fdfc65-rr5pc_fc9d977a-1414-4de2-97eb-cf9368d51887/prom-label-proxy/0.log" Jul 14 17:53:05.462435 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.462404 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf6fdfc65-rr5pc_fc9d977a-1414-4de2-97eb-cf9368d51887/kube-rbac-proxy-rules/0.log" Jul 14 17:53:05.485379 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:05.485347 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bf6fdfc65-rr5pc_fc9d977a-1414-4de2-97eb-cf9368d51887/kube-rbac-proxy-metrics/0.log" Jul 14 17:53:06.092954 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.092917 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24"] Jul 14 17:53:06.093395 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.093381 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad2189d-6b77-4f93-82b1-265811097ca5" containerName="revoke-keys" Jul 14 17:53:06.093442 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.093396 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad2189d-6b77-4f93-82b1-265811097ca5" containerName="revoke-keys" Jul 14 17:53:06.093442 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.093418 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="419a7613-11d9-4eb7-ad30-b8ed279d61ed" containerName="maas-api" Jul 14 17:53:06.093442 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.093423 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="419a7613-11d9-4eb7-ad30-b8ed279d61ed" containerName="maas-api" Jul 14 17:53:06.093558 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.093495 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad2189d-6b77-4f93-82b1-265811097ca5" containerName="revoke-keys" Jul 14 17:53:06.093558 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.093505 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="419a7613-11d9-4eb7-ad30-b8ed279d61ed" containerName="maas-api" Jul 14 17:53:06.098519 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.098491 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.101805 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.101774 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bv25z\"/\"openshift-service-ca.crt\"" Jul 14 17:53:06.102319 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.101806 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bv25z\"/\"default-dockercfg-lld7c\"" Jul 14 17:53:06.102864 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.102841 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bv25z\"/\"kube-root-ca.crt\"" Jul 14 17:53:06.109122 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.109089 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24"] Jul 14 17:53:06.194071 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.194020 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-podres\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.194279 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.194086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-sys\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.194279 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.194162 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-proc\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.194279 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.194202 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z58ht\" (UniqueName: \"kubernetes.io/projected/64b138bb-093f-4a1f-ab68-c3f60b962c27-kube-api-access-z58ht\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.194436 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.194282 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-lib-modules\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295203 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295167 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-proc\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z58ht\" (UniqueName: \"kubernetes.io/projected/64b138bb-093f-4a1f-ab68-c3f60b962c27-kube-api-access-z58ht\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295266 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-lib-modules\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-podres\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-sys\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295305 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-proc\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295401 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-lib-modules\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295442 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-sys\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.295672 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.295458 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/64b138bb-093f-4a1f-ab68-c3f60b962c27-podres\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.305731 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.305703 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z58ht\" (UniqueName: \"kubernetes.io/projected/64b138bb-093f-4a1f-ab68-c3f60b962c27-kube-api-access-z58ht\") pod \"perf-node-gather-daemonset-h8r24\" (UID: \"64b138bb-093f-4a1f-ab68-c3f60b962c27\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.410698 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.410605 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:06.561358 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.561328 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24"] Jul 14 17:53:06.562639 ip-10-0-128-240 kubenswrapper[2558]: W0714 17:53:06.562611 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod64b138bb_093f_4a1f_ab68_c3f60b962c27.slice/crio-cfc5acb60421d07caf50b5640ab57c8ee40a9f796d2f6c795a7964f224da563a WatchSource:0}: Error finding container cfc5acb60421d07caf50b5640ab57c8ee40a9f796d2f6c795a7964f224da563a: Status 404 returned error can't find the container with id cfc5acb60421d07caf50b5640ab57c8ee40a9f796d2f6c795a7964f224da563a Jul 14 17:53:06.768942 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:06.768884 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-bf2qg_e812b93e-638c-429e-93cd-c4fa57fc3f73/networking-console-plugin/0.log" Jul 14 17:53:07.232287 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:07.232252 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" event={"ID":"64b138bb-093f-4a1f-ab68-c3f60b962c27","Type":"ContainerStarted","Data":"8f70be92c1df2f56b95a764594d7f91cc86e433bb5a2b83dd900ad1cd2290de4"} Jul 14 17:53:07.232486 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:07.232310 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" event={"ID":"64b138bb-093f-4a1f-ab68-c3f60b962c27","Type":"ContainerStarted","Data":"cfc5acb60421d07caf50b5640ab57c8ee40a9f796d2f6c795a7964f224da563a"} Jul 14 17:53:07.232486 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:07.232331 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:07.253061 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:07.253004 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" podStartSLOduration=1.25298572 podStartE2EDuration="1.25298572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:53:07.250859528 +0000 UTC m=+2889.820270320" watchObservedRunningTime="2026-07-14 17:53:07.25298572 +0000 UTC m=+2889.822396492" Jul 14 17:53:07.362143 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:07.362109 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/1.log" Jul 14 17:53:07.369934 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:07.369903 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lpj7g_c2be7952-ab97-43f0-a8b5-1f227c22b10c/console-operator/2.log" Jul 14 17:53:08.580084 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:08.580059 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-g2gjj_e4845b9f-e512-48e6-b8d4-2ae25652b0d0/volume-data-source-validator/0.log" Jul 14 17:53:09.678512 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:09.678478 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7g9zk_5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f/dns/0.log" Jul 14 17:53:09.701750 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:09.701722 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7g9zk_5f1bb4bb-de02-41c7-9fc1-8b7d38f22b5f/kube-rbac-proxy/0.log" Jul 14 17:53:09.896927 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:09.896897 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pkfmh_2bc646b1-d627-4d8f-a261-d84275abdb68/dns-node-resolver/0.log" Jul 14 17:53:10.487470 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:10.487307 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2fd85_6992d303-6bd8-485d-b532-f782c6809cbc/node-ca/0.log" Jul 14 17:53:11.772992 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:11.772963 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vsq7s_3920de5e-8ecc-4746-b3e5-edb2d9d3d688/discovery/0.log" Jul 14 17:53:12.036026 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:12.035926 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7497bbf74b-2lkjk_fef52ed7-0972-4f2e-8cc0-2780b7f59855/router/0.log" Jul 14 17:53:12.745358 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:12.745330 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c4hcb_2f955fd3-d01a-4775-8f30-a6d37287c0e0/serve-healthcheck-canary/0.log" Jul 14 17:53:13.248922 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:13.248895 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-h8r24" Jul 14 17:53:13.320332 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:13.320296 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-zlgcw_df83cbc3-0758-4506-8e21-6641839d5f8b/insights-operator/0.log" Jul 14 17:53:13.321227 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:13.321203 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-zlgcw_df83cbc3-0758-4506-8e21-6641839d5f8b/insights-operator/1.log" Jul 14 17:53:13.357117 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:13.357081 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hflz4_f9eb7757-e64d-4550-bc5f-e5c6841bfb52/kube-rbac-proxy/0.log" Jul 14 17:53:13.384637 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:13.384599 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hflz4_f9eb7757-e64d-4550-bc5f-e5c6841bfb52/exporter/0.log" Jul 14 17:53:13.422922 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:13.422889 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hflz4_f9eb7757-e64d-4550-bc5f-e5c6841bfb52/extractor/0.log" Jul 14 17:53:16.857869 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:16.857831 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-xpk77_4b907c15-81f1-454a-b936-6d5ae0d886e5/manager/0.log" Jul 14 17:53:16.885017 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:16.884987 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-xpk77_4b907c15-81f1-454a-b936-6d5ae0d886e5/copy-manifests/0.log" Jul 14 17:53:16.904156 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:16.904119 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vsq7s_3920de5e-8ecc-4746-b3e5-edb2d9d3d688/discovery/0.log" Jul 14 17:53:17.180677 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:17.180568 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7497bbf74b-2lkjk_fef52ed7-0972-4f2e-8cc0-2780b7f59855/router/0.log" Jul 14 17:53:18.223089 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:18.223062 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5684c55548-5xmzm_4b90d5c7-ee54-41cc-9ef5-9c07023112eb/manager/0.log" Jul 14 17:53:18.680875 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:18.680841 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vsq7s_3920de5e-8ecc-4746-b3e5-edb2d9d3d688/discovery/0.log" Jul 14 17:53:18.925933 ip-10-0-128-240 kubenswrapper[2558]: I0714 17:53:18.925903 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7497bbf74b-2lkjk_fef52ed7-0972-4f2e-8cc0-2780b7f59855/router/0.log"