Jul 06 17:38:24.879576 ip-10-0-138-108 systemd[1]: Starting Kubernetes Kubelet... Jul 06 17:38:26.528356 ip-10-0-138-108 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 17:38:26.528356 ip-10-0-138-108 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 17:38:26.528356 ip-10-0-138-108 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 17:38:26.528356 ip-10-0-138-108 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 17:38:26.528356 ip-10-0-138-108 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 17:38:26.529290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.529197 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 17:38:26.533476 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533420 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:38:26.533476 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533470 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:38:26.533476 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533476 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:38:26.533476 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533481 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533487 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533499 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533507 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533512 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533517 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533528 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533533 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533541 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533548 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533554 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533559 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533564 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533571 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533576 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533582 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533586 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533591 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533601 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533606 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:38:26.533708 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533610 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533614 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533619 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533624 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533629 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533633 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533638 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533643 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533647 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533652 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533657 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533667 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533672 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533678 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533684 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533689 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533694 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533699 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533704 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:38:26.534532 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533709 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533714 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533719 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533724 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533735 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533740 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533745 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533754 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533760 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533765 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533771 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533776 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533781 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533786 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533791 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533796 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533807 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533812 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533818 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:38:26.535078 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533822 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533827 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533832 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533837 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533842 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533846 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533851 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533856 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533868 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533873 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533884 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533889 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533893 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533901 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533905 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533910 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533914 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533919 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533924 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533929 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:38:26.535601 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533935 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533944 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533949 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533954 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533959 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533964 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533969 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533974 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533979 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533983 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533988 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533993 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.533998 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.534002 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.534015 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.534023 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.534028 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.534033 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535161 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535172 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:38:26.536093 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535176 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535180 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535184 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535187 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535190 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535192 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535195 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535198 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535202 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535204 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535207 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535210 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535213 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535215 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535218 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535221 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535224 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535226 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535229 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535232 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:38:26.536643 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535235 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535237 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535240 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535242 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535246 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535249 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535251 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535254 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535257 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535260 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535262 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535266 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535269 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535272 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535275 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535277 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535280 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535282 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535285 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:38:26.537199 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535290 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535295 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535298 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535302 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535305 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535308 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535311 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535314 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535317 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535321 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535325 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535328 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535331 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535334 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535337 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535339 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535343 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535347 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535350 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:38:26.537710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535353 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535377 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535382 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535386 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535390 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535396 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535399 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535402 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535406 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535409 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535412 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535415 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535418 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535420 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535423 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535426 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535429 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535432 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535434 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535437 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:38:26.538236 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535440 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535443 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535446 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535450 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535453 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535455 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535458 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535461 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535463 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535466 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535469 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535472 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535474 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535477 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535480 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535482 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535485 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535488 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535491 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535973 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:38:26.538781 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535979 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535983 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535986 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535989 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535993 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535996 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.535999 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536002 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536005 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536008 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536011 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536014 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536017 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536020 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536023 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536026 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536029 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536032 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536035 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536038 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:38:26.539289 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536041 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536044 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536047 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536049 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536052 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536055 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536058 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536061 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536063 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536066 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536069 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536072 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536075 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536077 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536080 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536083 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536086 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536088 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536091 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536094 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:38:26.539826 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536096 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536099 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536102 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536105 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536108 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536111 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536113 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536116 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536120 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536123 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536125 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536128 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536130 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536133 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536136 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536139 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536141 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536144 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536147 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:38:26.540352 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536149 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536154 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536158 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536161 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536164 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536167 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536171 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536174 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536177 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536181 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536185 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536190 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536194 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536197 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536200 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536203 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536206 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536209 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536212 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:38:26.540863 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536215 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536217 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536221 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536224 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536228 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536231 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536234 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536236 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536239 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536242 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536245 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536247 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536250 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536253 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536256 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536259 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536262 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536264 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536267 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.536270 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:38:26.541387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536347 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536355 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536361 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536386 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536393 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536398 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536404 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536410 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536413 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536417 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536420 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536424 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536427 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536430 2580 flags.go:64] FLAG: --cgroup-root="" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536433 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536437 2580 flags.go:64] FLAG: --client-ca-file="" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536441 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536444 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536449 2580 flags.go:64] FLAG: --cluster-domain="" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536452 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536456 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536459 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536462 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536467 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 17:38:26.541925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536470 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536474 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536478 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536481 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536484 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536487 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536491 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536494 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536499 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536502 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536505 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536508 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536512 2580 flags.go:64] FLAG: --enable-server="true" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536515 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536519 2580 flags.go:64] FLAG: --event-burst="100" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536523 2580 flags.go:64] FLAG: --event-qps="50" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536526 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536529 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536533 2580 flags.go:64] FLAG: --eviction-hard="" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536537 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536540 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536543 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536546 2580 flags.go:64] FLAG: --eviction-soft="" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536549 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536553 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 17:38:26.542557 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536556 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536560 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536563 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536566 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536569 2580 flags.go:64] FLAG: --feature-gates="" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536573 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536579 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536583 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536586 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536589 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536592 2580 flags.go:64] FLAG: --help="false" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536596 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536599 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536603 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536606 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536609 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536613 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536616 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536619 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536622 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536625 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536628 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536631 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536634 2580 flags.go:64] FLAG: --kube-reserved="" Jul 06 17:38:26.543242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536638 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536640 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536644 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536647 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536650 2580 flags.go:64] FLAG: --lock-file="" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536653 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536657 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536660 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536665 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536676 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536680 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536682 2580 flags.go:64] FLAG: --logging-format="text" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536686 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536689 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536694 2580 flags.go:64] FLAG: --manifest-url="" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536697 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536707 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536710 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536714 2580 flags.go:64] FLAG: --max-pods="110" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536717 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536720 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536724 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536727 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536730 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536733 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 17:38:26.543978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536736 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536744 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536748 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536751 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536754 2580 flags.go:64] FLAG: --pod-cidr="" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536757 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536764 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536767 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536770 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536773 2580 flags.go:64] FLAG: --port="10250" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536777 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536780 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ac039cd2067a621e" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536784 2580 flags.go:64] FLAG: --qos-reserved="" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536787 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536790 2580 flags.go:64] FLAG: --register-node="true" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536793 2580 flags.go:64] FLAG: --register-with-taints="" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536797 2580 flags.go:64] FLAG: --registry-burst="10" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536800 2580 flags.go:64] FLAG: --registry-qps="5" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536803 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536806 2580 flags.go:64] FLAG: --reserved-memory="" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536810 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536817 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536820 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536823 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536826 2580 flags.go:64] FLAG: --runonce="false" Jul 06 17:38:26.544662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536829 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536832 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536835 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536838 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536841 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536845 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536848 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536852 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536855 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536858 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536861 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536864 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536867 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536870 2580 flags.go:64] FLAG: --system-cgroups="" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536873 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536878 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536881 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536884 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536890 2580 flags.go:64] FLAG: --tls-min-version="" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536893 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536896 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536899 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536902 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536905 2580 flags.go:64] FLAG: --v="2" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536909 2580 flags.go:64] FLAG: --version="false" Jul 06 17:38:26.545315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536920 2580 flags.go:64] FLAG: --vmodule="" Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536925 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.536928 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537074 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537078 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537081 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537085 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537087 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537090 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537122 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537126 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537130 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537133 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537137 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537140 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537143 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537146 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537149 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537152 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537155 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:38:26.545975 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537157 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537160 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537163 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537166 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537169 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537172 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537174 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537177 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537180 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537183 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537185 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537188 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537191 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537200 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537204 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537209 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537212 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537216 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537219 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537221 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:38:26.546513 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537224 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537227 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537229 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537232 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537236 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537241 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537246 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537250 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537254 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537257 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537260 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537264 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537267 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537270 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537272 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537275 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537278 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537280 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537283 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:38:26.547043 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537286 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537289 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537291 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537294 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537297 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537299 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537302 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537305 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537312 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537315 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537318 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537321 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537323 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537326 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537329 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537331 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537334 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537337 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537340 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537342 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:38:26.547612 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537346 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537348 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537351 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537354 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537356 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537359 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537377 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537381 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537384 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537387 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537390 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537392 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537395 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537398 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537401 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537404 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537406 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537409 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537412 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537414 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:38:26.548128 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537420 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537423 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.537426 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.538191 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.545815 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.545836 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.545859 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.545865 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545931 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545939 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545943 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545946 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545950 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545954 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545957 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:38:26.548763 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545960 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545963 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545967 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545970 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545973 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545976 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545979 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545982 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545986 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545989 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545993 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545996 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.545999 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546002 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546005 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546009 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546013 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546016 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546020 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546022 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:38:26.549157 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546026 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546029 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546034 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546037 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546040 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546042 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546045 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546048 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546051 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546054 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546057 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546059 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546062 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546065 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546068 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546070 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546073 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546076 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546079 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546082 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:38:26.549719 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546085 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546087 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546090 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546093 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546095 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546098 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546101 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546103 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546106 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546109 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546111 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546114 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546117 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546120 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546124 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546127 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546130 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546133 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546136 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:38:26.550274 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546138 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546141 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546144 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546147 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546149 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546152 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546155 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546157 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546160 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546163 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546166 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546168 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546171 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546174 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546177 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546180 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546184 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546187 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546191 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546194 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:38:26.550798 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546197 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546200 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546203 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546206 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546209 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546212 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546215 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546218 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546221 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546227 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546230 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546233 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546236 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.546242 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546383 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:38:26.551324 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546389 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546393 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546397 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546400 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546403 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546405 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546408 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546411 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546414 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546416 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546419 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546422 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546425 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546427 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546430 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546433 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546435 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546438 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546441 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546444 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:38:26.551786 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546447 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546449 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546452 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546457 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546461 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546464 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546467 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546471 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546474 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546477 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546480 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546483 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546485 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546488 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546492 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546496 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546499 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546502 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546505 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:38:26.552301 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546508 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546511 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546514 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546517 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546520 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546523 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546526 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546529 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546532 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546534 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546537 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546540 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546543 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546546 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546548 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546551 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546554 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546557 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546560 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:38:26.552815 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546563 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546565 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546569 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546571 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546574 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546577 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546580 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546582 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546585 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546589 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546592 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546595 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546598 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546602 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546605 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546608 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546611 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546613 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546616 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546619 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:38:26.553309 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546622 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546625 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546627 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546630 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546633 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546636 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546639 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546642 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546645 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546647 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546650 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546653 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546656 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546659 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546661 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546665 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546667 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546670 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546673 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:38:26.553851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:26.546676 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:38:26.554387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.546681 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 17:38:26.554387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.546825 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 17:38:26.554387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.549868 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 17:38:26.554387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.551255 2580 server.go:1019] "Starting client certificate rotation" Jul 06 17:38:26.554387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.551368 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 17:38:26.554387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.551407 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 17:38:26.554387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.554287 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 17:38:26.555618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.555594 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 17:38:26.566531 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.566508 2580 log.go:25] "Validated CRI v1 runtime API" Jul 06 17:38:26.569665 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.569647 2580 log.go:25] "Validated CRI v1 image API" Jul 06 17:38:26.570434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.570418 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 17:38:26.573411 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.573391 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9c8161c5-a4c0-4adb-8dda-2e471a769d04:/dev/nvme0n1p3 fa5c353c-40d7-4280-b1dd-a1e60affd355:/dev/nvme0n1p4] Jul 06 17:38:26.573467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.573411 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 17:38:26.579805 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.579675 2580 manager.go:216] Machine: {Timestamp:2026-07-06 17:38:26.578352994 +0000 UTC m=+0.328720564 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091048 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2664c668b48526ca677b100e231843 SystemUUID:ec2664c6-68b4-8526-ca67-7b100e231843 BootID:00007645-c6f4-4662-a9a6-363ca031e29a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:27:3a:33:8a:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:27:3a:33:8a:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:33:34:b8:92:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 17:38:26.579805 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.579795 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 17:38:26.579958 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.579885 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 17:38:26.583100 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.583069 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 17:38:26.583289 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.583105 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-108.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 17:38:26.583842 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.583830 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 17:38:26.583900 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.583847 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 17:38:26.583900 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.583859 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 17:38:26.584521 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.584508 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 17:38:26.584757 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.584739 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 17:38:26.586109 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.586094 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 17:38:26.589737 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.589722 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 06 17:38:26.589878 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.589868 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 17:38:26.589923 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.589902 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 06 17:38:26.589923 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.589916 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 17:38:26.589989 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.589941 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 17:38:26.589989 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.589957 2580 kubelet.go:398] "Adding apiserver pod source" Jul 06 17:38:26.589989 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.589971 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 17:38:26.590376 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.590355 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 17:38:26.590414 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.590391 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 17:38:26.592801 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.592780 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 17:38:26.595169 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.595152 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 17:38:26.595251 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.595185 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 17:38:26.596419 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596406 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 17:38:26.596467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596425 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 17:38:26.596467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596432 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 17:38:26.596467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596438 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 17:38:26.596467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596447 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 17:38:26.596467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596453 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 17:38:26.596467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596461 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 17:38:26.596467 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596467 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 17:38:26.596683 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596475 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 17:38:26.596683 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596481 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 17:38:26.596683 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596490 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 17:38:26.596683 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.596496 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 17:38:26.597207 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.597195 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 17:38:26.597207 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.597207 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 17:38:26.600905 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.600890 2580 server.go:1268] "Started kubelet" Jul 06 17:38:26.601015 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.600978 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 17:38:26.601176 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.601088 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 17:38:26.601855 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.601834 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 17:38:26.602057 ip-10-0-138-108 systemd[1]: Started Kubernetes Kubelet. Jul 06 17:38:26.603389 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.603299 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 17:38:26.603389 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.603308 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-108.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 17:38:26.607215 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.607178 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 17:38:26.607653 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.607624 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-108.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 17:38:26.608532 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.608508 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 17:38:26.608769 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.608753 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 06 17:38:26.611328 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.610206 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-108.ec2.internal.18bfc405c9c29a83 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-108.ec2.internal,UID:ip-10-0-138-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-108.ec2.internal,},FirstTimestamp:2026-07-06 17:38:26.600860291 +0000 UTC m=+0.351227861,LastTimestamp:2026-07-06 17:38:26.600860291 +0000 UTC m=+0.351227861,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-108.ec2.internal,}" Jul 06 17:38:26.614420 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.614404 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 17:38:26.614493 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.614424 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 17:38:26.614609 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.614590 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:26.615466 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615445 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 17:38:26.615466 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615446 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 17:38:26.615596 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615478 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 17:38:26.615596 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615518 2580 factory.go:55] Registering systemd factory Jul 06 17:38:26.615596 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615537 2580 factory.go:223] Registration of the systemd container factory successfully Jul 06 17:38:26.615596 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615569 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 06 17:38:26.615596 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615582 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 06 17:38:26.615826 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615760 2580 factory.go:153] Registering CRI-O factory Jul 06 17:38:26.615826 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615775 2580 factory.go:223] Registration of the crio container factory successfully Jul 06 17:38:26.615918 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615834 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 17:38:26.615918 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615875 2580 factory.go:103] Registering Raw factory Jul 06 17:38:26.615918 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.615888 2580 manager.go:1196] Started watching for new ooms in manager Jul 06 17:38:26.616564 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.616544 2580 manager.go:349] Starting recovery of all containers Jul 06 17:38:26.619759 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.619730 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 17:38:26.619870 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.619844 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-108.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 17:38:26.629124 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.628948 2580 manager.go:354] Recovery completed Jul 06 17:38:26.633577 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.633555 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k28b4" Jul 06 17:38:26.633938 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.633925 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:38:26.635922 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.635901 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:38:26.636004 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.635938 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:38:26.636004 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.635950 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:38:26.636522 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.636509 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 17:38:26.636522 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.636520 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 17:38:26.636650 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.636526 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:38:26.636650 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.636537 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 06 17:38:26.637262 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.637245 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:38:26.637337 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.637274 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:38:26.637337 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.637286 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:38:26.637856 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.637844 2580 policy_none.go:49] "None policy: Start" Jul 06 17:38:26.637889 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.637860 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 17:38:26.637889 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.637871 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 17:38:26.638342 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.638275 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-108.ec2.internal.18bfc405cbd9958f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-108.ec2.internal,UID:ip-10-0-138-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-108.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-108.ec2.internal,},FirstTimestamp:2026-07-06 17:38:26.635920783 +0000 UTC m=+0.386288358,LastTimestamp:2026-07-06 17:38:26.635920783 +0000 UTC m=+0.386288358,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-108.ec2.internal,}" Jul 06 17:38:26.638879 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.638865 2580 policy_none.go:47] "Start" Jul 06 17:38:26.641610 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.641583 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k28b4" Jul 06 17:38:26.677051 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.677028 2580 manager.go:337] "Starting Device Plugin manager" Jul 06 17:38:26.677175 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.677069 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 17:38:26.677175 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.677079 2580 server.go:87] "Starting device plugin registration server" Jul 06 17:38:26.677444 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.677429 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 17:38:26.677514 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.677451 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 17:38:26.677564 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.677524 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 17:38:26.677632 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.677621 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 17:38:26.677694 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.677633 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 17:38:26.678200 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.678176 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 17:38:26.678277 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.678238 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:26.759789 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.759738 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 17:38:26.762158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.761099 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 17:38:26.762158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.761144 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 17:38:26.762158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.761173 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 17:38:26.762158 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.761218 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 17:38:26.763417 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.763398 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:38:26.778580 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.778515 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:38:26.779512 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.779492 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:38:26.779566 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.779534 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:38:26.779566 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.779549 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:38:26.779638 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.779574 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.789422 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.789404 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.789559 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.789431 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-108.ec2.internal\": node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:26.807621 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.807593 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:26.862392 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.862345 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal"] Jul 06 17:38:26.862553 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.862428 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:38:26.864198 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.864178 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:38:26.864303 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.864211 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:38:26.864303 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.864221 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:38:26.866586 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.866572 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:38:26.866712 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.866696 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.866779 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.866727 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:38:26.867383 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.867347 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:38:26.867383 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.867374 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:38:26.867383 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.867386 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:38:26.867517 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.867347 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:38:26.867517 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.867449 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:38:26.867517 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.867465 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:38:26.869888 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.869870 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.869957 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.869911 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:38:26.870657 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.870639 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:38:26.870739 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.870667 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:38:26.870739 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.870680 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:38:26.894273 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.894246 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-108.ec2.internal\" not found" node="ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.899144 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.899123 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-108.ec2.internal\" not found" node="ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.907982 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:26.907958 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:26.917506 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.917483 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ec4acbf852e2b07ece08ecc1022c7c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-108.ec2.internal\" (UID: \"b7ec4acbf852e2b07ece08ecc1022c7c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.917606 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.917526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/305d9cf3b9e0fa073efd5151ba0d6bd1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal\" (UID: \"305d9cf3b9e0fa073efd5151ba0d6bd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:26.917606 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:26.917551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/305d9cf3b9e0fa073efd5151ba0d6bd1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal\" (UID: \"305d9cf3b9e0fa073efd5151ba0d6bd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.008149 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:27.008107 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:27.018107 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.018070 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ec4acbf852e2b07ece08ecc1022c7c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-108.ec2.internal\" (UID: \"b7ec4acbf852e2b07ece08ecc1022c7c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.018210 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.018107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ec4acbf852e2b07ece08ecc1022c7c-config\") pod \"kube-apiserver-proxy-ip-10-0-138-108.ec2.internal\" (UID: \"b7ec4acbf852e2b07ece08ecc1022c7c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.018210 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.018148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/305d9cf3b9e0fa073efd5151ba0d6bd1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal\" (UID: \"305d9cf3b9e0fa073efd5151ba0d6bd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.018210 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.018169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/305d9cf3b9e0fa073efd5151ba0d6bd1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal\" (UID: \"305d9cf3b9e0fa073efd5151ba0d6bd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.018210 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.018196 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/305d9cf3b9e0fa073efd5151ba0d6bd1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal\" (UID: \"305d9cf3b9e0fa073efd5151ba0d6bd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.018371 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.018250 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/305d9cf3b9e0fa073efd5151ba0d6bd1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal\" (UID: \"305d9cf3b9e0fa073efd5151ba0d6bd1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.109285 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:27.109196 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:27.205001 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.204960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.205001 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.204987 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.210430 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:27.210099 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:27.313788 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:27.313744 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:27.421519 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:27.421446 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:27.522434 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:27.522401 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:27.550608 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.550573 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 17:38:27.551020 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.550771 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 17:38:27.623025 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:27.622991 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-108.ec2.internal\" not found" Jul 06 17:38:27.644466 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.644411 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 17:33:27 +0000 UTC" deadline="2028-04-03 10:32:44.330025466 +0000 UTC" Jul 06 17:38:27.644466 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.644457 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15280h54m16.685571895s" Jul 06 17:38:27.678605 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.678523 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:38:27.716877 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.716846 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.730354 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.730326 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 17:38:27.732560 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.732541 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" Jul 06 17:38:27.739470 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.739450 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 17:38:27.742704 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.742686 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:38:27.852410 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:27.852379 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:38:28.412856 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.412822 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod305d9cf3b9e0fa073efd5151ba0d6bd1.slice/crio-90a338ccdc19b02d35de0e7e4c7b1b7df88930dd6955b588b7d6769dbd73d87d WatchSource:0}: Error finding container 90a338ccdc19b02d35de0e7e4c7b1b7df88930dd6955b588b7d6769dbd73d87d: Status 404 returned error can't find the container with id 90a338ccdc19b02d35de0e7e4c7b1b7df88930dd6955b588b7d6769dbd73d87d Jul 06 17:38:28.413556 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.413531 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ec4acbf852e2b07ece08ecc1022c7c.slice/crio-84233f16aaf8461869eb4b073f6edb0bf38ba9a33dd51d30f25d0cef03ffa8a4 WatchSource:0}: Error finding container 84233f16aaf8461869eb4b073f6edb0bf38ba9a33dd51d30f25d0cef03ffa8a4: Status 404 returned error can't find the container with id 84233f16aaf8461869eb4b073f6edb0bf38ba9a33dd51d30f25d0cef03ffa8a4 Jul 06 17:38:28.418301 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.418285 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 17:38:28.593024 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.592926 2580 apiserver.go:52] "Watching apiserver" Jul 06 17:38:28.604105 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.603433 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 17:38:28.605139 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.605103 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-wm6tm","openshift-multus/multus-qk5dl","openshift-multus/network-metrics-daemon-nch8h","openshift-network-diagnostics/network-check-target-qlnlr","kube-system/global-pull-secret-syncer-64lqb","kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48","openshift-cluster-node-tuning-operator/tuned-zkc4w","openshift-network-operator/iptables-alerter-4qcl2","openshift-ovn-kubernetes/ovnkube-node-w5lpr","kube-system/konnectivity-agent-lkf6j","openshift-dns/node-resolver-d5x7w","openshift-image-registry/node-ca-jlngt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal"] Jul 06 17:38:28.609268 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.609248 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.611456 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.611434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.612583 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.612555 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bvc2m\"" Jul 06 17:38:28.612583 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.612572 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 17:38:28.612718 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.612563 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 17:38:28.612718 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.612571 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 17:38:28.613709 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.613689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:28.613797 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.613766 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:28.614485 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.614464 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 17:38:28.614485 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.614477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d8f58\"" Jul 06 17:38:28.614485 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.614472 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 17:38:28.614673 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.614540 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 17:38:28.614914 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.614897 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 17:38:28.616056 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.616036 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:28.616144 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.616107 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:28.618432 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.618414 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.618526 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.618507 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:28.620724 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.620708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.623035 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.623018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.623695 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.623675 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:38:28.623785 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.623736 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 17:38:28.623785 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.623736 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tdvgl\"" Jul 06 17:38:28.625399 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.625354 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.626008 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.625992 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 17:38:28.626177 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.626165 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:38:28.626325 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.626311 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 17:38:28.626521 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.626506 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hvx7m\"" Jul 06 17:38:28.627827 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.627810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.628224 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628208 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7b8qq\"" Jul 06 17:38:28.628315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628229 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-device-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.628315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.628315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-serviceca\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.628315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxz8h\" (UniqueName: \"kubernetes.io/projected/d2e94037-82c6-43ee-bc44-d391a411b64d-kube-api-access-vxz8h\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:28.628315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zpnd\" (UniqueName: \"kubernetes.io/projected/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-kube-api-access-6zpnd\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628368 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-host\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9p4x\" (UniqueName: \"kubernetes.io/projected/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-kube-api-access-x9p4x\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628458 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-socket-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628495 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-registration-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.628571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-sys-fs\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.629010 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628739 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 17:38:28.629010 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628788 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 17:38:28.629010 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.628790 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 17:38:28.629681 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.629655 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 17:38:28.630360 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.630332 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5x2vf\"" Jul 06 17:38:28.630446 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.630371 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.630568 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.630553 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 17:38:28.633065 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.633045 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.633593 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.633325 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 17:38:28.633593 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.633356 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 17:38:28.633593 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.633375 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 17:38:28.633593 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.633533 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kbcgn\"" Jul 06 17:38:28.633593 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.633537 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 17:38:28.633593 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.633567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 17:38:28.635500 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.635475 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 17:38:28.635738 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.635720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.636079 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.636057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n7npx\"" Jul 06 17:38:28.639594 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.637494 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 17:38:28.639594 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.637580 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 17:38:28.639594 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.638803 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hswb7\"" Jul 06 17:38:28.639746 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.639635 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 17:38:28.640126 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.640103 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 17:38:28.649774 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.649757 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d7jzp" Jul 06 17:38:28.657954 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.657937 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d7jzp" Jul 06 17:38:28.716560 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.716525 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 17:38:28.729514 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-system-cni-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.729514 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729515 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-hostroot\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.729715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-systemd\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.729715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovnkube-script-lib\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.729715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-sys\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.729715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-registration-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.729715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-ovn\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.729715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729688 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-registration-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.729715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-var-lib-kubelet\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-device-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-serviceca\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729802 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-etc-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729805 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-device-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovn-node-metrics-cert\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729898 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-hosts-file\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-tmp-dir\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.729974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729954 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/085b4edd-6175-4c47-b96f-f079c9da977b-dbus\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.729994 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b89k4\" (UniqueName: \"kubernetes.io/projected/eed02498-3532-46ef-bed9-9467491ec2fc-kube-api-access-b89k4\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-modprobe-d\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-tmp\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9p4x\" (UniqueName: \"kubernetes.io/projected/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-kube-api-access-x9p4x\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-systemd-units\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cnibin\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730185 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-socket-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-cnibin\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-k8s-cni-cncf-io\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730238 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-node-log\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxz8h\" (UniqueName: \"kubernetes.io/projected/d2e94037-82c6-43ee-bc44-d391a411b64d-kube-api-access-vxz8h\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730294 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-serviceca\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-os-release\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730377 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-socket-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-run-netns\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-cni-bin\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/085b4edd-6175-4c47-b96f-f079c9da977b-kubelet-config\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-kubernetes\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysctl-d\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-kubelet\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730567 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-log-socket\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-env-overrides\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730613 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-system-cni-dir\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730631 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-os-release\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.730806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-tuned\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-sys-fs\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730740 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-sys-fs\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-cni-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-netns\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-cni-bin\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730843 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eed02498-3532-46ef-bed9-9467491ec2fc-host-slash\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plvsk\" (UniqueName: \"kubernetes.io/projected/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-kube-api-access-plvsk\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysconfig\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.730976 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-systemd\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-lib-modules\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731034 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zpnd\" (UniqueName: \"kubernetes.io/projected/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-kube-api-access-6zpnd\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-cni-binary-copy\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.731551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-kubelet\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731165 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-daemon-config\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.731221 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-slash\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.731350 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:38:29.231270674 +0000 UTC m=+2.981638274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731373 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-var-lib-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731406 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-socket-dir-parent\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731429 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-cni-netd\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731521 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mj7d\" (UniqueName: \"kubernetes.io/projected/df4e23f6-8e9f-4390-923e-7c97baffa4e3-kube-api-access-7mj7d\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhcd\" (UniqueName: \"kubernetes.io/projected/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-kube-api-access-cqhcd\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-host\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-cni-multus\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.732290 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-multus-certs\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eed02498-3532-46ef-bed9-9467491ec2fc-iptables-alerter-script\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731700 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq646\" (UniqueName: \"kubernetes.io/projected/35d04c92-1a0e-42ee-a63e-91dc5e35f966-kube-api-access-dq646\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dgbq\" (UniqueName: \"kubernetes.io/projected/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-kube-api-access-9dgbq\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731786 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-etc-kubernetes\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cni-binary-copy\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731839 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-run\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-host\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-conf-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovnkube-config\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-host\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5f611c0-3ab7-4bd8-9332-06ac4922a8c3-agent-certs\") pod \"konnectivity-agent-lkf6j\" (UID: \"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3\") " pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731978 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5f611c0-3ab7-4bd8-9332-06ac4922a8c3-konnectivity-ca\") pod \"konnectivity-agent-lkf6j\" (UID: \"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3\") " pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.732903 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.731995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysctl-conf\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.740705 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.740686 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:38:28.740705 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.740704 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:38:28.740861 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.740714 2580 projected.go:196] Error preparing data for projected volume kube-api-access-npth9 for pod openshift-network-diagnostics/network-check-target-qlnlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:28.740861 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.740723 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 17:38:28.740861 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.740770 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9 podName:1eb50574-dc4c-47cd-b1c1-7593c50ba6c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:38:29.240756071 +0000 UTC m=+2.991123627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-npth9" (UniqueName: "kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9") pod "network-check-target-qlnlr" (UID: "1eb50574-dc4c-47cd-b1c1-7593c50ba6c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:28.744227 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.744206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9p4x\" (UniqueName: \"kubernetes.io/projected/9d3c8ac9-7048-4cfa-9d32-5a1b54164436-kube-api-access-x9p4x\") pod \"aws-ebs-csi-driver-node-w8q48\" (UID: \"9d3c8ac9-7048-4cfa-9d32-5a1b54164436\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.744227 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.744224 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zpnd\" (UniqueName: \"kubernetes.io/projected/e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3-kube-api-access-6zpnd\") pod \"node-ca-jlngt\" (UID: \"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3\") " pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.744424 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.744207 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxz8h\" (UniqueName: \"kubernetes.io/projected/d2e94037-82c6-43ee-bc44-d391a411b64d-kube-api-access-vxz8h\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:28.765965 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.765912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" event={"ID":"305d9cf3b9e0fa073efd5151ba0d6bd1","Type":"ContainerStarted","Data":"90a338ccdc19b02d35de0e7e4c7b1b7df88930dd6955b588b7d6769dbd73d87d"} Jul 06 17:38:28.767131 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.767101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" event={"ID":"b7ec4acbf852e2b07ece08ecc1022c7c","Type":"ContainerStarted","Data":"84233f16aaf8461869eb4b073f6edb0bf38ba9a33dd51d30f25d0cef03ffa8a4"} Jul 06 17:38:28.795839 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.795803 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:38:28.832953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.832915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plvsk\" (UniqueName: \"kubernetes.io/projected/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-kube-api-access-plvsk\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.832953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.832956 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysconfig\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.833206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.832982 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-systemd\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.833206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-lib-modules\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.833206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833072 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysconfig\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.833206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-systemd\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.833206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833153 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-lib-modules\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.833206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-cni-binary-copy\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-kubelet\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-daemon-config\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833261 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-kubelet\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-slash\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-var-lib-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-socket-dir-parent\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833387 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-slash\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833390 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-var-lib-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833424 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-socket-dir-parent\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-cni-netd\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833489 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833502 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-cni-netd\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mj7d\" (UniqueName: \"kubernetes.io/projected/df4e23f6-8e9f-4390-923e-7c97baffa4e3-kube-api-access-7mj7d\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.833548 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhcd\" (UniqueName: \"kubernetes.io/projected/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-kube-api-access-cqhcd\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-host\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-cni-multus\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833702 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-cni-multus\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833717 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-host\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-multus-certs\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eed02498-3532-46ef-bed9-9467491ec2fc-iptables-alerter-script\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-multus-certs\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq646\" (UniqueName: \"kubernetes.io/projected/35d04c92-1a0e-42ee-a63e-91dc5e35f966-kube-api-access-dq646\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833836 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dgbq\" (UniqueName: \"kubernetes.io/projected/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-kube-api-access-9dgbq\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833869 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-cni-binary-copy\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833880 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-daemon-config\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833920 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-etc-kubernetes\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cni-binary-copy\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833970 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-etc-kubernetes\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.833979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-run\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.834442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834024 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-conf-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovnkube-config\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834070 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-conf-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5f611c0-3ab7-4bd8-9332-06ac4922a8c3-agent-certs\") pod \"konnectivity-agent-lkf6j\" (UID: \"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3\") " pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834027 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-run\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5f611c0-3ab7-4bd8-9332-06ac4922a8c3-konnectivity-ca\") pod \"konnectivity-agent-lkf6j\" (UID: \"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3\") " pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysctl-conf\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-system-cni-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-hostroot\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-systemd\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovnkube-script-lib\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834235 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-sys\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834297 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-ovn\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysctl-conf\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834359 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-system-cni-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834399 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-systemd\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834414 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-var-lib-kubelet\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.835229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834447 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-run-ovn\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834449 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eed02498-3532-46ef-bed9-9467491ec2fc-iptables-alerter-script\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-hostroot\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834462 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-sys\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-var-lib-kubelet\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834490 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cni-binary-copy\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834500 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-etc-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834543 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-etc-openvswitch\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovn-node-metrics-cert\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-hosts-file\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-tmp-dir\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/085b4edd-6175-4c47-b96f-f079c9da977b-dbus\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b89k4\" (UniqueName: \"kubernetes.io/projected/eed02498-3532-46ef-bed9-9467491ec2fc-kube-api-access-b89k4\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834676 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovnkube-config\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834713 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-hosts-file\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-modprobe-d\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-tmp\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.836158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-systemd-units\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834837 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-modprobe-d\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cnibin\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-cnibin\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-k8s-cni-cncf-io\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834925 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5f611c0-3ab7-4bd8-9332-06ac4922a8c3-konnectivity-ca\") pod \"konnectivity-agent-lkf6j\" (UID: \"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3\") " pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-node-log\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834950 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovnkube-script-lib\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-node-log\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-tmp-dir\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.834983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-os-release\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-run-netns\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-cni-bin\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835041 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-systemd-units\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.835064 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835106 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837097 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:28.835128 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret podName:085b4edd-6175-4c47-b96f-f079c9da977b nodeName:}" failed. No retries permitted until 2026-07-06 17:38:29.335109025 +0000 UTC m=+3.085476583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret") pod "global-pull-secret-syncer-64lqb" (UID: "085b4edd-6175-4c47-b96f-f079c9da977b") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835149 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-run-netns\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835166 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cnibin\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835189 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-cni-bin\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835217 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/085b4edd-6175-4c47-b96f-f079c9da977b-dbus\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835251 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-os-release\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/085b4edd-6175-4c47-b96f-f079c9da977b-kubelet-config\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-kubernetes\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835301 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-k8s-cni-cncf-io\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835222 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-cnibin\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysctl-d\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-kubelet\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835362 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/085b4edd-6175-4c47-b96f-f079c9da977b-kubelet-config\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-log-socket\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-env-overrides\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835435 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-kubernetes\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.837962 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-system-cni-dir\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835455 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35d04c92-1a0e-42ee-a63e-91dc5e35f966-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835466 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-log-socket\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-os-release\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835498 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-sysctl-d\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835469 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df4e23f6-8e9f-4390-923e-7c97baffa4e3-host-kubelet\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-os-release\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835549 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-tuned\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835520 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-system-cni-dir\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-cni-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-netns\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-cni-bin\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835666 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-multus-cni-dir\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835702 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-run-netns\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eed02498-3532-46ef-bed9-9467491ec2fc-host-slash\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835708 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-host-var-lib-cni-bin\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.838546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835712 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eed02498-3532-46ef-bed9-9467491ec2fc-host-slash\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.839034 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835666 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35d04c92-1a0e-42ee-a63e-91dc5e35f966-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.839034 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.835934 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df4e23f6-8e9f-4390-923e-7c97baffa4e3-env-overrides\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.839034 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.837327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df4e23f6-8e9f-4390-923e-7c97baffa4e3-ovn-node-metrics-cert\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.839034 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.837428 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5f611c0-3ab7-4bd8-9332-06ac4922a8c3-agent-certs\") pod \"konnectivity-agent-lkf6j\" (UID: \"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3\") " pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.839034 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.837509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-tmp\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.839034 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.837963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-etc-tuned\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.842549 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.842516 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plvsk\" (UniqueName: \"kubernetes.io/projected/cddf9fb1-4a06-4bd4-939a-c9b59ca24f30-kube-api-access-plvsk\") pod \"multus-qk5dl\" (UID: \"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30\") " pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.842549 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.842542 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dgbq\" (UniqueName: \"kubernetes.io/projected/4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb-kube-api-access-9dgbq\") pod \"tuned-zkc4w\" (UID: \"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb\") " pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.842741 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.842527 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhcd\" (UniqueName: \"kubernetes.io/projected/d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931-kube-api-access-cqhcd\") pod \"node-resolver-d5x7w\" (UID: \"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931\") " pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.842931 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.842912 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq646\" (UniqueName: \"kubernetes.io/projected/35d04c92-1a0e-42ee-a63e-91dc5e35f966-kube-api-access-dq646\") pod \"multus-additional-cni-plugins-wm6tm\" (UID: \"35d04c92-1a0e-42ee-a63e-91dc5e35f966\") " pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.843404 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.843341 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mj7d\" (UniqueName: \"kubernetes.io/projected/df4e23f6-8e9f-4390-923e-7c97baffa4e3-kube-api-access-7mj7d\") pod \"ovnkube-node-w5lpr\" (UID: \"df4e23f6-8e9f-4390-923e-7c97baffa4e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.843805 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.843786 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b89k4\" (UniqueName: \"kubernetes.io/projected/eed02498-3532-46ef-bed9-9467491ec2fc-kube-api-access-b89k4\") pod \"iptables-alerter-4qcl2\" (UID: \"eed02498-3532-46ef-bed9-9467491ec2fc\") " pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.920596 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.920561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" Jul 06 17:38:28.925227 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.925198 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jlngt" Jul 06 17:38:28.932247 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.932217 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3c8ac9_7048_4cfa_9d32_5a1b54164436.slice/crio-4ca35ca8b0468e050c579101598b5ccad7b97d434b5770e0d2e36b58ba48204d WatchSource:0}: Error finding container 4ca35ca8b0468e050c579101598b5ccad7b97d434b5770e0d2e36b58ba48204d: Status 404 returned error can't find the container with id 4ca35ca8b0468e050c579101598b5ccad7b97d434b5770e0d2e36b58ba48204d Jul 06 17:38:28.933854 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.933828 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8eddfc3_a5cb_4dfc_b9c9_9b9e3f2c1ab3.slice/crio-c39cb5fc6f37dd6440841638e6b48bdfbb9866a85c26764a1144927c0695aad9 WatchSource:0}: Error finding container c39cb5fc6f37dd6440841638e6b48bdfbb9866a85c26764a1144927c0695aad9: Status 404 returned error can't find the container with id c39cb5fc6f37dd6440841638e6b48bdfbb9866a85c26764a1144927c0695aad9 Jul 06 17:38:28.933854 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.933849 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" Jul 06 17:38:28.942100 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.942063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4qcl2" Jul 06 17:38:28.945217 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.945192 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c09dca2_c10f_4e45_b5a1_0c3b6b9167fb.slice/crio-e53fbba736bb63dab5994a9f6f7798c43f12de505b3bbb815ee846699595d92d WatchSource:0}: Error finding container e53fbba736bb63dab5994a9f6f7798c43f12de505b3bbb815ee846699595d92d: Status 404 returned error can't find the container with id e53fbba736bb63dab5994a9f6f7798c43f12de505b3bbb815ee846699595d92d Jul 06 17:38:28.947464 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.947438 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" Jul 06 17:38:28.953629 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.953610 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qk5dl" Jul 06 17:38:28.960001 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.959972 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:28.965501 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.965481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:28.969267 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.969239 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcddf9fb1_4a06_4bd4_939a_c9b59ca24f30.slice/crio-07a21398cc79ff52ab5de28b3fe255193b17ea95707eb653052c80ab6cb6cbf0 WatchSource:0}: Error finding container 07a21398cc79ff52ab5de28b3fe255193b17ea95707eb653052c80ab6cb6cbf0: Status 404 returned error can't find the container with id 07a21398cc79ff52ab5de28b3fe255193b17ea95707eb653052c80ab6cb6cbf0 Jul 06 17:38:28.970036 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:28.970017 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d5x7w" Jul 06 17:38:28.971710 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.971681 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d04c92_1a0e_42ee_a63e_91dc5e35f966.slice/crio-51c828e838dcb8f7b05904528cb3faf8ed3e89d5f79998b75090e05c2669ec6c WatchSource:0}: Error finding container 51c828e838dcb8f7b05904528cb3faf8ed3e89d5f79998b75090e05c2669ec6c: Status 404 returned error can't find the container with id 51c828e838dcb8f7b05904528cb3faf8ed3e89d5f79998b75090e05c2669ec6c Jul 06 17:38:28.981776 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.981748 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f611c0_3ab7_4bd8_9332_06ac4922a8c3.slice/crio-e5e5a467123a2c0994ec0e320e7e550d08d4059427784ccd336918d06a46ecb0 WatchSource:0}: Error finding container e5e5a467123a2c0994ec0e320e7e550d08d4059427784ccd336918d06a46ecb0: Status 404 returned error can't find the container with id e5e5a467123a2c0994ec0e320e7e550d08d4059427784ccd336918d06a46ecb0 Jul 06 17:38:28.985089 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:38:28.985066 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e72a07_00c8_4c7e_ab0f_a21b8e0ae931.slice/crio-d6ca70da6a47fbc61c36ae4dc515ec1de8bd86008bed26da2c5bcb00efea623c WatchSource:0}: Error finding container d6ca70da6a47fbc61c36ae4dc515ec1de8bd86008bed26da2c5bcb00efea623c: Status 404 returned error can't find the container with id d6ca70da6a47fbc61c36ae4dc515ec1de8bd86008bed26da2c5bcb00efea623c Jul 06 17:38:29.239047 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.238952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:29.239212 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.239168 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:29.239284 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.239257 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:38:30.239234729 +0000 UTC m=+3.989602302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:29.339581 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.339541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:29.339800 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.339609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:29.339800 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.339738 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:29.339921 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.339822 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret podName:085b4edd-6175-4c47-b96f-f079c9da977b nodeName:}" failed. No retries permitted until 2026-07-06 17:38:30.339799546 +0000 UTC m=+4.090167107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret") pod "global-pull-secret-syncer-64lqb" (UID: "085b4edd-6175-4c47-b96f-f079c9da977b") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:29.339921 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.339750 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:38:29.339921 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.339861 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:38:29.339921 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.339878 2580 projected.go:196] Error preparing data for projected volume kube-api-access-npth9 for pod openshift-network-diagnostics/network-check-target-qlnlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:29.339921 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.339918 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9 podName:1eb50574-dc4c-47cd-b1c1-7593c50ba6c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:38:30.339906055 +0000 UTC m=+4.090273614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-npth9" (UniqueName: "kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9") pod "network-check-target-qlnlr" (UID: "1eb50574-dc4c-47cd-b1c1-7593c50ba6c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:29.661099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.661055 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 17:33:29 +0000 UTC" deadline="2028-03-20 19:31:37.150323994 +0000 UTC" Jul 06 17:38:29.661099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.661095 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14953h53m7.489233774s" Jul 06 17:38:29.762052 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.762015 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:29.762263 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.762177 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:29.762682 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.762659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:29.762798 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:29.762773 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:29.798338 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.798240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerStarted","Data":"51c828e838dcb8f7b05904528cb3faf8ed3e89d5f79998b75090e05c2669ec6c"} Jul 06 17:38:29.818354 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.818290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jlngt" event={"ID":"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3","Type":"ContainerStarted","Data":"c39cb5fc6f37dd6440841638e6b48bdfbb9866a85c26764a1144927c0695aad9"} Jul 06 17:38:29.827283 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.827144 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" event={"ID":"9d3c8ac9-7048-4cfa-9d32-5a1b54164436","Type":"ContainerStarted","Data":"4ca35ca8b0468e050c579101598b5ccad7b97d434b5770e0d2e36b58ba48204d"} Jul 06 17:38:29.848345 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.848287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lkf6j" event={"ID":"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3","Type":"ContainerStarted","Data":"e5e5a467123a2c0994ec0e320e7e550d08d4059427784ccd336918d06a46ecb0"} Jul 06 17:38:29.867405 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.867339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4qcl2" event={"ID":"eed02498-3532-46ef-bed9-9467491ec2fc","Type":"ContainerStarted","Data":"ae95c54cab6b7f3cc470a75efcde4e055d773197c0e411d300b157305762b2a1"} Jul 06 17:38:29.886957 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.886861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qk5dl" event={"ID":"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30","Type":"ContainerStarted","Data":"07a21398cc79ff52ab5de28b3fe255193b17ea95707eb653052c80ab6cb6cbf0"} Jul 06 17:38:29.893273 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.893230 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" event={"ID":"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb","Type":"ContainerStarted","Data":"e53fbba736bb63dab5994a9f6f7798c43f12de505b3bbb815ee846699595d92d"} Jul 06 17:38:29.916416 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.916372 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d5x7w" event={"ID":"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931","Type":"ContainerStarted","Data":"d6ca70da6a47fbc61c36ae4dc515ec1de8bd86008bed26da2c5bcb00efea623c"} Jul 06 17:38:29.936104 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:29.936064 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"6892e7d2acbd1d9ef45c49eaa6b72bcbf2a6f3d01be64d254280a563d9f0f44b"} Jul 06 17:38:30.248656 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:30.248552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:30.248829 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.248769 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:30.248894 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.248839 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:38:32.248818285 +0000 UTC m=+5.999185841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:30.349491 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:30.349419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:30.349491 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:30.349497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:30.349758 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.349592 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:30.349758 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.349651 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:38:30.349758 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.349663 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret podName:085b4edd-6175-4c47-b96f-f079c9da977b nodeName:}" failed. No retries permitted until 2026-07-06 17:38:32.34964285 +0000 UTC m=+6.100010425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret") pod "global-pull-secret-syncer-64lqb" (UID: "085b4edd-6175-4c47-b96f-f079c9da977b") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:30.349758 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.349667 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:38:30.349758 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.349681 2580 projected.go:196] Error preparing data for projected volume kube-api-access-npth9 for pod openshift-network-diagnostics/network-check-target-qlnlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:30.349758 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.349718 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9 podName:1eb50574-dc4c-47cd-b1c1-7593c50ba6c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:38:32.34970724 +0000 UTC m=+6.100074810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-npth9" (UniqueName: "kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9") pod "network-check-target-qlnlr" (UID: "1eb50574-dc4c-47cd-b1c1-7593c50ba6c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:30.662347 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:30.662224 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 17:33:29 +0000 UTC" deadline="2028-03-17 13:50:49.154319674 +0000 UTC" Jul 06 17:38:30.662347 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:30.662291 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14876h12m18.492032561s" Jul 06 17:38:30.765505 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:30.764835 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:30.765505 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:30.764983 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:31.762576 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:31.762541 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:31.763101 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:31.762593 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:31.763101 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:31.762702 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:31.763101 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:31.762836 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:32.267125 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:32.267089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:32.267337 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.267286 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:32.267421 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.267369 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:38:36.267347243 +0000 UTC m=+10.017714814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:32.368190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:32.368298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.368442 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.368511 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret podName:085b4edd-6175-4c47-b96f-f079c9da977b nodeName:}" failed. No retries permitted until 2026-07-06 17:38:36.368490973 +0000 UTC m=+10.118858530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret") pod "global-pull-secret-syncer-64lqb" (UID: "085b4edd-6175-4c47-b96f-f079c9da977b") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.368938 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.368959 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.368971 2580 projected.go:196] Error preparing data for projected volume kube-api-access-npth9 for pod openshift-network-diagnostics/network-check-target-qlnlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:32.369061 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.369019 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9 podName:1eb50574-dc4c-47cd-b1c1-7593c50ba6c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:38:36.369003393 +0000 UTC m=+10.119370953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-npth9" (UniqueName: "kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9") pod "network-check-target-qlnlr" (UID: "1eb50574-dc4c-47cd-b1c1-7593c50ba6c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:32.761780 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:32.761695 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:32.761961 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:32.761840 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:33.762327 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:33.762216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:33.762900 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:33.762397 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:33.762900 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:33.762222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:33.762900 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:33.762689 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:34.762012 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:34.761802 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:34.762012 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:34.761976 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:35.761571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:35.761535 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:35.761947 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:35.761531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:35.761947 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:35.761683 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:35.761947 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:35.761739 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:36.304473 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:36.304355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:36.304688 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.304542 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:36.304688 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.304627 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:38:44.30460439 +0000 UTC m=+18.054972009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:36.405424 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:36.405166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:36.405424 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:36.405263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:36.405424 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.405400 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:38:36.405424 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.405430 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:38:36.405773 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.405446 2580 projected.go:196] Error preparing data for projected volume kube-api-access-npth9 for pod openshift-network-diagnostics/network-check-target-qlnlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:36.405773 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.405515 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9 podName:1eb50574-dc4c-47cd-b1c1-7593c50ba6c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:38:44.405494044 +0000 UTC m=+18.155861605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-npth9" (UniqueName: "kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9") pod "network-check-target-qlnlr" (UID: "1eb50574-dc4c-47cd-b1c1-7593c50ba6c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:36.405773 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.405402 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:36.405773 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.405608 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret podName:085b4edd-6175-4c47-b96f-f079c9da977b nodeName:}" failed. No retries permitted until 2026-07-06 17:38:44.405593499 +0000 UTC m=+18.155961071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret") pod "global-pull-secret-syncer-64lqb" (UID: "085b4edd-6175-4c47-b96f-f079c9da977b") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:36.762709 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:36.762209 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:36.762709 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:36.762332 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:37.761388 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:37.761360 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:37.761547 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:37.761494 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:37.761674 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:37.761648 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:37.761791 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:37.761739 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:38.762514 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:38.762432 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:38.762910 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:38.762582 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:39.761690 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:39.761637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:39.761887 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:39.761769 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:39.762197 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:39.761637 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:39.762197 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:39.762164 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:40.762302 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:40.762236 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:40.762752 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:40.762393 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:41.761630 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:41.761590 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:41.761630 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:41.761633 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:41.761856 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:41.761754 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:41.761904 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:41.761867 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:42.761834 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:42.761795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:42.762338 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:42.761933 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:43.762146 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:43.762114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:43.762146 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:43.762136 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:43.762689 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:43.762255 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:43.762689 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:43.762383 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:44.367830 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:44.367796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:44.368052 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.367923 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:44.368052 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.367992 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:39:00.367975189 +0000 UTC m=+34.118342765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:38:44.468591 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:44.468557 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:44.468828 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:44.468612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:44.468828 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.468713 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:44.468828 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.468724 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:38:44.468828 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.468744 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:38:44.468828 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.468756 2580 projected.go:196] Error preparing data for projected volume kube-api-access-npth9 for pod openshift-network-diagnostics/network-check-target-qlnlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:44.468828 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.468788 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret podName:085b4edd-6175-4c47-b96f-f079c9da977b nodeName:}" failed. No retries permitted until 2026-07-06 17:39:00.468770028 +0000 UTC m=+34.219137603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret") pod "global-pull-secret-syncer-64lqb" (UID: "085b4edd-6175-4c47-b96f-f079c9da977b") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:38:44.468828 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.468808 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9 podName:1eb50574-dc4c-47cd-b1c1-7593c50ba6c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:39:00.46879869 +0000 UTC m=+34.219166244 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-npth9" (UniqueName: "kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9") pod "network-check-target-qlnlr" (UID: "1eb50574-dc4c-47cd-b1c1-7593c50ba6c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:38:44.761879 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:44.761860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:44.761980 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:44.761961 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:45.761826 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:45.761787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:45.762280 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:45.761787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:45.762280 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:45.761923 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:45.762280 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:45.762009 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:46.762304 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:46.762268 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:46.762788 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:46.762389 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:47.761804 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:47.761765 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:47.761995 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:47.761902 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:47.761995 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:47.761961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:47.762115 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:47.762074 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:48.761879 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:48.761844 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:48.762458 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:48.761977 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:49.761850 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:49.761812 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:49.762017 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:49.761822 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:49.762017 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:49.761950 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:49.762478 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:49.762027 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:49.983561 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:49.983518 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" event={"ID":"b7ec4acbf852e2b07ece08ecc1022c7c","Type":"ContainerStarted","Data":"7b27b4409d28b661aab6055d8b00f5641b19edf54ce8af164a1587d9696fd738"} Jul 06 17:38:49.985078 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:49.985048 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qk5dl" event={"ID":"cddf9fb1-4a06-4bd4-939a-c9b59ca24f30","Type":"ContainerStarted","Data":"4bbbc92ac47ad2b55893d0d0f4cc993ffba1f1fb719664a35dd818138748e45d"} Jul 06 17:38:49.986457 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:49.986436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" event={"ID":"4c09dca2-c10f-4e45-b5a1-0c3b6b9167fb","Type":"ContainerStarted","Data":"03c55ff2f59d9a635d2374323d79abac63295dc19c6a90b740e223787a8a1e93"} Jul 06 17:38:49.998050 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:49.998003 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-108.ec2.internal" podStartSLOduration=21.997991118 podStartE2EDuration="21.997991118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:49.997671727 +0000 UTC m=+23.748039302" watchObservedRunningTime="2026-07-06 17:38:49.997991118 +0000 UTC m=+23.748358694" Jul 06 17:38:50.013399 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.013308 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zkc4w" podStartSLOduration=2.702172384 podStartE2EDuration="23.013274723s" totalImagesPullingTime="20.311102339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:50.012779607 +0000 UTC m=+23.763147183" watchObservedRunningTime="2026-07-06 17:38:50.013274723 +0000 UTC m=+23.763642299" Jul 06 17:38:50.033465 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.033417 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qk5dl" podStartSLOduration=2.465188093 podStartE2EDuration="23.033400146s" totalImagesPullingTime="20.568212053s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:50.032909785 +0000 UTC m=+23.783277360" watchObservedRunningTime="2026-07-06 17:38:50.033400146 +0000 UTC m=+23.783767722" Jul 06 17:38:50.761810 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.761630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:50.761958 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:50.761918 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:50.990423 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.990173 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d5x7w" event={"ID":"d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931","Type":"ContainerStarted","Data":"b71dc19a1c4b222b6e0fa6838e363621b760bfb57f8104014e69dce87b2bc929"} Jul 06 17:38:50.992838 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.992818 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:38:50.993101 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.993077 2580 generic.go:356] "Generic (PLEG): container finished" podID="df4e23f6-8e9f-4390-923e-7c97baffa4e3" containerID="b7ca4b5b614ef1b263ece03e1ecf2d4843f16a2b2c5a4f2c3081094ac0f7f4b1" exitCode=1 Jul 06 17:38:50.993230 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.993106 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"049e033428cb9638d896639215140f533a180c93ba069e2a7e390211b82c5cad"} Jul 06 17:38:50.993230 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.993187 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"97830dfb0daccd6d0c635a182925d7a2fdf2d5aebbe265a5131863ffdb40137d"} Jul 06 17:38:50.993371 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.993238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"7bac6942f857d6aefe3d83b1cca7e8e5e21417f490e5d38b0c9bbfe0a6373106"} Jul 06 17:38:50.993371 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.993302 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"929da93ef61636aefbe456708a0406e29efc1a9ac1a3361012aa3649b5cbc332"} Jul 06 17:38:50.993371 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.993352 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerDied","Data":"b7ca4b5b614ef1b263ece03e1ecf2d4843f16a2b2c5a4f2c3081094ac0f7f4b1"} Jul 06 17:38:50.993499 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.993403 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"45d02ed603b59d22167c6b0f7c262f0f620c1ed67e181e66ddddc71560ebf9ef"} Jul 06 17:38:50.994459 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.994438 2580 generic.go:356] "Generic (PLEG): container finished" podID="35d04c92-1a0e-42ee-a63e-91dc5e35f966" containerID="4dbea8818f2d11d2afb62e9a7a98843c103c1d11b21ba838e7649072ac059756" exitCode=0 Jul 06 17:38:50.994549 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.994522 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerDied","Data":"4dbea8818f2d11d2afb62e9a7a98843c103c1d11b21ba838e7649072ac059756"} Jul 06 17:38:50.995865 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.995798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jlngt" event={"ID":"e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3","Type":"ContainerStarted","Data":"44a73cbe4992e2d74702da83921e28e4b5a78a600c30e2a038545ca4e7582461"} Jul 06 17:38:50.997241 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.997221 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" event={"ID":"9d3c8ac9-7048-4cfa-9d32-5a1b54164436","Type":"ContainerStarted","Data":"0433537a512e4838a76d913b5679bd197322afcd75243488657c45dfc95f490f"} Jul 06 17:38:50.998512 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.998491 2580 generic.go:356] "Generic (PLEG): container finished" podID="305d9cf3b9e0fa073efd5151ba0d6bd1" containerID="ad6d8ccd34da41f75e93247aae92798b2b8e20ef61bab2ee83bf5df1f4d986c2" exitCode=0 Jul 06 17:38:50.998640 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.998553 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" event={"ID":"305d9cf3b9e0fa073efd5151ba0d6bd1","Type":"ContainerDied","Data":"ad6d8ccd34da41f75e93247aae92798b2b8e20ef61bab2ee83bf5df1f4d986c2"} Jul 06 17:38:50.999753 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:50.999731 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lkf6j" event={"ID":"d5f611c0-3ab7-4bd8-9332-06ac4922a8c3","Type":"ContainerStarted","Data":"dbd972970967fa4cf2963310f3f49c39d271d35f9752c9ceaba6e6324e7a15d4"} Jul 06 17:38:51.001093 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:51.001070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4qcl2" event={"ID":"eed02498-3532-46ef-bed9-9467491ec2fc","Type":"ContainerStarted","Data":"6e2dfeafdd49cde5ae5117ef9c3694d0404e1e058b8cb06f6149d7db979f7122"} Jul 06 17:38:51.033105 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:51.033062 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d5x7w" podStartSLOduration=3.7407446159999997 podStartE2EDuration="24.033048226s" totalImagesPullingTime="20.29230361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:51.014176464 +0000 UTC m=+24.764544040" watchObservedRunningTime="2026-07-06 17:38:51.033048226 +0000 UTC m=+24.783415801" Jul 06 17:38:51.050123 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:51.050011 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jlngt" podStartSLOduration=3.742591063 podStartE2EDuration="24.049994428s" totalImagesPullingTime="20.307403365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:51.049758874 +0000 UTC m=+24.800126450" watchObservedRunningTime="2026-07-06 17:38:51.049994428 +0000 UTC m=+24.800362003" Jul 06 17:38:51.067855 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:51.067798 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lkf6j" podStartSLOduration=3.780126113 podStartE2EDuration="24.067781115s" totalImagesPullingTime="20.287655002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:51.066930729 +0000 UTC m=+24.817298334" watchObservedRunningTime="2026-07-06 17:38:51.067781115 +0000 UTC m=+24.818148691" Jul 06 17:38:51.084141 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:51.084093 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4qcl2" podStartSLOduration=3.852313827 podStartE2EDuration="24.084077074s" totalImagesPullingTime="20.231763247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:51.083667939 +0000 UTC m=+24.834035514" watchObservedRunningTime="2026-07-06 17:38:51.084077074 +0000 UTC m=+24.834444650" Jul 06 17:38:51.761657 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:51.761594 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:51.761883 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:51.761605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:51.761883 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:51.761727 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:51.761883 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:51.761813 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:52.009077 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:52.009040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" event={"ID":"305d9cf3b9e0fa073efd5151ba0d6bd1","Type":"ContainerStarted","Data":"bf99373ba29237efe041169cd89977d5743756c1452d7935275847fcb89f2859"} Jul 06 17:38:52.024516 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:52.024391 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-108.ec2.internal" podStartSLOduration=24.024377451 podStartE2EDuration="24.024377451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:52.02434305 +0000 UTC m=+25.774710626" watchObservedRunningTime="2026-07-06 17:38:52.024377451 +0000 UTC m=+25.774745027" Jul 06 17:38:52.407501 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:52.407474 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 17:38:52.690704 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:52.690568 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T17:38:52.407494101Z","UUID":"e70c75ae-56fb-4068-8b50-c1cd9af5baca","Handler":null,"Name":"","Endpoint":""} Jul 06 17:38:52.691972 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:52.691951 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 17:38:52.692077 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:52.691988 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 17:38:52.761816 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:52.761775 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:52.762030 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:52.761909 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:53.013965 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:53.013929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" event={"ID":"9d3c8ac9-7048-4cfa-9d32-5a1b54164436","Type":"ContainerStarted","Data":"d240a5428fc1eacb0dc794c79425bc1e6fe4379eedd1f36aa4baa0c29392cc7e"} Jul 06 17:38:53.017614 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:53.017588 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:38:53.018561 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:53.018260 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"2e531dabe7a909d4e12e6ab766d049c1bd40f07192856b48de35aa9bd1bfb8a7"} Jul 06 17:38:53.623966 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:53.623591 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:53.624227 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:53.624190 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:53.761424 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:53.761403 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:53.761555 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:53.761521 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:53.761618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:53.761559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:53.761670 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:53.761648 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:54.022607 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:54.022570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" event={"ID":"9d3c8ac9-7048-4cfa-9d32-5a1b54164436","Type":"ContainerStarted","Data":"ef163a1e288f085d59b0842292b233fa766c2b1639f56f6737ec8df6d9e4cdeb"} Jul 06 17:38:54.023089 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:54.022583 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 17:38:54.039936 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:54.039890 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8q48" podStartSLOduration=4.045997623 podStartE2EDuration="27.039875537s" totalImagesPullingTime="22.993877914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:54.038366029 +0000 UTC m=+27.788733619" watchObservedRunningTime="2026-07-06 17:38:54.039875537 +0000 UTC m=+27.790243168" Jul 06 17:38:54.762317 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:54.762272 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:54.762481 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:54.762400 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:55.762454 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:55.762188 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:55.763342 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:55.762212 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:55.763342 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:55.762541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:55.763342 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:55.762629 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:56.029776 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.029712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:38:56.030092 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.030068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"957e678430bda3dc8b8f1ac8a4ec76744b9e243f9f132f69ca8c3d3c90a1cb24"} Jul 06 17:38:56.030346 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.030323 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:56.030501 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.030488 2580 scope.go:117] "RemoveContainer" containerID="b7ca4b5b614ef1b263ece03e1ecf2d4843f16a2b2c5a4f2c3081094ac0f7f4b1" Jul 06 17:38:56.046209 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.046186 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:56.081902 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.081878 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:56.082090 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.082017 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 17:38:56.082592 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.082500 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lkf6j" Jul 06 17:38:56.762934 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:56.762906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:56.763533 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:56.763005 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:57.036471 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.036403 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:38:57.036836 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.036811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" event={"ID":"df4e23f6-8e9f-4390-923e-7c97baffa4e3","Type":"ContainerStarted","Data":"010e9b56af0332ea6b3988aad9f0b677c6596fbfe1005c7c816ec5f4136b9b01"} Jul 06 17:38:57.037206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.037181 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:57.037313 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.037283 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:57.054869 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.054841 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:38:57.064685 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.064635 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" podStartSLOduration=8.913984046 podStartE2EDuration="30.064617125s" totalImagesPullingTime="21.150633079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:38:57.062793657 +0000 UTC m=+30.813161233" watchObservedRunningTime="2026-07-06 17:38:57.064617125 +0000 UTC m=+30.814984702" Jul 06 17:38:57.078109 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.078071 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nch8h"] Jul 06 17:38:57.078268 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.078222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:57.078402 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:57.078361 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:57.081346 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.081243 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-64lqb"] Jul 06 17:38:57.081487 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.081350 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:57.081487 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:57.081447 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:57.092876 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.092845 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qlnlr"] Jul 06 17:38:57.093022 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:57.092984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:57.093107 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:57.093087 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:58.762477 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:58.762441 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:38:58.762838 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:58.762454 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:38:58.762838 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:58.762547 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:38:58.762838 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:58.762635 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:38:58.762838 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:58.762684 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:38:58.762838 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:38:58.762756 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:38:59.044668 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:59.044636 2580 generic.go:356] "Generic (PLEG): container finished" podID="35d04c92-1a0e-42ee-a63e-91dc5e35f966" containerID="231824d0543d5bb0dfcdd97938a8dcfb7d838c35bb82d9eb4b6b6f1642a23fe0" exitCode=0 Jul 06 17:38:59.044753 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:38:59.044712 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerDied","Data":"231824d0543d5bb0dfcdd97938a8dcfb7d838c35bb82d9eb4b6b6f1642a23fe0"} Jul 06 17:39:00.048878 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.048688 2580 generic.go:356] "Generic (PLEG): container finished" podID="35d04c92-1a0e-42ee-a63e-91dc5e35f966" containerID="eca074ba4703d0a70517cb9653457548e7bff2d2108f50f5d1460a0d0e1a6170" exitCode=0 Jul 06 17:39:00.049413 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.048747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerDied","Data":"eca074ba4703d0a70517cb9653457548e7bff2d2108f50f5d1460a0d0e1a6170"} Jul 06 17:39:00.387254 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.387156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:39:00.387421 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.387281 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:39:00.387421 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.387353 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:39:32.387338279 +0000 UTC m=+66.137705837 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:39:00.488341 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.488305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:39:00.488481 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.488350 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:39:00.488481 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.488448 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:39:00.488542 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.488513 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret podName:085b4edd-6175-4c47-b96f-f079c9da977b nodeName:}" failed. No retries permitted until 2026-07-06 17:39:32.488497987 +0000 UTC m=+66.238865570 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret") pod "global-pull-secret-syncer-64lqb" (UID: "085b4edd-6175-4c47-b96f-f079c9da977b") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:39:00.488542 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.488452 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:39:00.488635 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.488553 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:39:00.488635 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.488566 2580 projected.go:196] Error preparing data for projected volume kube-api-access-npth9 for pod openshift-network-diagnostics/network-check-target-qlnlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:39:00.488635 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.488614 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9 podName:1eb50574-dc4c-47cd-b1c1-7593c50ba6c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:39:32.488599055 +0000 UTC m=+66.238966614 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-npth9" (UniqueName: "kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9") pod "network-check-target-qlnlr" (UID: "1eb50574-dc4c-47cd-b1c1-7593c50ba6c9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:39:00.762132 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.762095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:39:00.762316 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.762095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:39:00.762316 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.762236 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64lqb" podUID="085b4edd-6175-4c47-b96f-f079c9da977b" Jul 06 17:39:00.762316 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:00.762095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:39:00.762490 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.762323 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qlnlr" podUID="1eb50574-dc4c-47cd-b1c1-7593c50ba6c9" Jul 06 17:39:00.762490 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:00.762413 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nch8h" podUID="d2e94037-82c6-43ee-bc44-d391a411b64d" Jul 06 17:39:01.053166 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:01.053132 2580 generic.go:356] "Generic (PLEG): container finished" podID="35d04c92-1a0e-42ee-a63e-91dc5e35f966" containerID="2c4c2b5f4691eea77f74989ab2d29f22d5d237af51da3336abd69abbd83b1995" exitCode=0 Jul 06 17:39:01.053659 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:01.053178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerDied","Data":"2c4c2b5f4691eea77f74989ab2d29f22d5d237af51da3336abd69abbd83b1995"} Jul 06 17:39:02.266022 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.265999 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-108.ec2.internal" event="NodeReady" Jul 06 17:39:02.266372 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.266140 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 17:39:02.299112 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.299083 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65dc7bfff9-glg9j"] Jul 06 17:39:02.301478 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.301458 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.304240 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.304219 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 17:39:02.304383 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.304240 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 17:39:02.304527 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.304514 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 17:39:02.305031 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.305017 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p5ww8\"" Jul 06 17:39:02.309103 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.309085 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lkl8n"] Jul 06 17:39:02.310934 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.310913 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 17:39:02.311871 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.311855 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jvl6d"] Jul 06 17:39:02.311980 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.311964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.314533 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.314508 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 17:39:02.314533 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.314523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-75j6z\"" Jul 06 17:39:02.314727 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.314521 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 17:39:02.314876 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.314858 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.318482 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.318179 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-46kbs\"" Jul 06 17:39:02.318552 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.318530 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 17:39:02.318552 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.318537 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 17:39:02.318644 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.318625 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 17:39:02.318830 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.318811 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65dc7bfff9-glg9j"] Jul 06 17:39:02.320627 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.320608 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lkl8n"] Jul 06 17:39:02.323385 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.323364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jvl6d"] Jul 06 17:39:02.403706 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6999\" (UniqueName: \"kubernetes.io/projected/465e675d-31d0-4272-b140-23a5e780df74-kube-api-access-n6999\") pod \"ingress-canary-jvl6d\" (UID: \"465e675d-31d0-4272-b140-23a5e780df74\") " pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.403916 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403726 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-bound-sa-token\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.403916 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.403916 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-tmp-dir\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.403916 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.403916 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2884e0ee-2837-48cf-abfe-5e98ff81e79a-ca-trust-extracted\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.404165 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk4np\" (UniqueName: \"kubernetes.io/projected/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-kube-api-access-vk4np\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.404165 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.403998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-trusted-ca\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.404165 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.404066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-config-volume\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.404165 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.404104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/465e675d-31d0-4272-b140-23a5e780df74-cert\") pod \"ingress-canary-jvl6d\" (UID: \"465e675d-31d0-4272-b140-23a5e780df74\") " pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.404382 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.404171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-image-registry-private-configuration\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.404382 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.404217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-installation-pull-secrets\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.404382 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.404236 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-certificates\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.404382 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.404256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-649g6\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-kube-api-access-649g6\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.442889 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.442842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:02.504899 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.504818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-installation-pull-secrets\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.504899 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.504860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-certificates\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.504899 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.504879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-649g6\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-kube-api-access-649g6\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.504899 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.504902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6999\" (UniqueName: \"kubernetes.io/projected/465e675d-31d0-4272-b140-23a5e780df74-kube-api-access-n6999\") pod \"ingress-canary-jvl6d\" (UID: \"465e675d-31d0-4272-b140-23a5e780df74\") " pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.504920 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-bound-sa-token\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.504941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.504967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-tmp-dir\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2884e0ee-2837-48cf-abfe-5e98ff81e79a-ca-trust-extracted\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk4np\" (UniqueName: \"kubernetes.io/projected/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-kube-api-access-vk4np\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-trusted-ca\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-config-volume\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505150 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/465e675d-31d0-4272-b140-23a5e780df74-cert\") pod \"ingress-canary-jvl6d\" (UID: \"465e675d-31d0-4272-b140-23a5e780df74\") " pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:02.505200 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:39:02.505217 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:02.505217 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:39:02.505734 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:02.505264 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:39:02.505734 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:02.505285 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:39:03.005263238 +0000 UTC m=+36.755630793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:39:02.505734 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:02.505327 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:39:03.005316751 +0000 UTC m=+36.755684309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:39:02.505734 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505185 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-image-registry-private-configuration\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.505933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505843 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-tmp-dir\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.505987 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.505970 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2884e0ee-2837-48cf-abfe-5e98ff81e79a-ca-trust-extracted\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.506228 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.506205 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-certificates\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.509260 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.506324 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-config-volume\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.509260 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.507025 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-trusted-ca\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.511260 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.511227 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-image-registry-private-configuration\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.511387 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.511263 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-installation-pull-secrets\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.511834 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.511804 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/465e675d-31d0-4272-b140-23a5e780df74-cert\") pod \"ingress-canary-jvl6d\" (UID: \"465e675d-31d0-4272-b140-23a5e780df74\") " pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.514093 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.514072 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6999\" (UniqueName: \"kubernetes.io/projected/465e675d-31d0-4272-b140-23a5e780df74-kube-api-access-n6999\") pod \"ingress-canary-jvl6d\" (UID: \"465e675d-31d0-4272-b140-23a5e780df74\") " pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.514176 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.514150 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-649g6\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-kube-api-access-649g6\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.517176 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.517151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-bound-sa-token\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:02.517251 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.517147 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk4np\" (UniqueName: \"kubernetes.io/projected/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-kube-api-access-vk4np\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:02.528039 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.528007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:02.627863 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.627827 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jvl6d" Jul 06 17:39:02.762135 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.762009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:39:02.762341 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.762143 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:39:02.762448 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.762006 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:39:02.765503 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.765240 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 17:39:02.765503 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.765247 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 17:39:02.765503 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.765308 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 17:39:02.765503 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.765350 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkf68\"" Jul 06 17:39:02.765503 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.765240 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 17:39:02.765503 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.765247 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wwzt2\"" Jul 06 17:39:02.766753 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:02.766734 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jvl6d"] Jul 06 17:39:02.767665 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:39:02.767640 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465e675d_31d0_4272_b140_23a5e780df74.slice/crio-034d7b39eee0033f777174ae29c6492b0a0f69e66661671b74357db7215ef230 WatchSource:0}: Error finding container 034d7b39eee0033f777174ae29c6492b0a0f69e66661671b74357db7215ef230: Status 404 returned error can't find the container with id 034d7b39eee0033f777174ae29c6492b0a0f69e66661671b74357db7215ef230 Jul 06 17:39:03.010895 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:03.010862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:03.011073 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:03.010921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:03.011073 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:03.011040 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:39:03.011196 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:03.011082 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:39:03.011196 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:03.011097 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:39:03.011196 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:03.011119 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:39:04.011096968 +0000 UTC m=+37.761464540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:39:03.011196 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:03.011153 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:39:04.01113704 +0000 UTC m=+37.761504610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:39:03.057911 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:03.057818 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jvl6d" event={"ID":"465e675d-31d0-4272-b140-23a5e780df74","Type":"ContainerStarted","Data":"034d7b39eee0033f777174ae29c6492b0a0f69e66661671b74357db7215ef230"} Jul 06 17:39:04.020612 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:04.020566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:04.021203 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:04.020627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:04.021203 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:04.020724 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:39:04.021203 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:04.020762 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:39:04.021203 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:04.020777 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:39:04.021203 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:04.020790 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:39:06.02077346 +0000 UTC m=+39.771141030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:39:04.021203 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:04.020824 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:39:06.020809569 +0000 UTC m=+39.771177122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:39:06.035406 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:06.035366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:06.035868 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:06.035431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:06.035868 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:06.035609 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:39:06.035868 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:06.035625 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:39:06.035868 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:06.035691 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:39:10.035668578 +0000 UTC m=+43.786036152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:39:06.036126 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:06.036105 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:39:06.036187 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:06.036169 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:39:10.036150982 +0000 UTC m=+43.786518556 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:39:06.066978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:06.066937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jvl6d" event={"ID":"465e675d-31d0-4272-b140-23a5e780df74","Type":"ContainerStarted","Data":"52811d3249f299819a4c1f5632cafb01b8622ebe2f6eebc71dd050380f327d74"} Jul 06 17:39:06.082097 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:06.082043 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jvl6d" podStartSLOduration=1.268601685 podStartE2EDuration="4.082025919s" totalImagesPullingTime="2.813424234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:39:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:39:06.081986361 +0000 UTC m=+39.832353938" watchObservedRunningTime="2026-07-06 17:39:06.082025919 +0000 UTC m=+39.832393475" Jul 06 17:39:07.922415 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:07.922180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:08.022205 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:08.022127 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:08.075157 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:08.075125 2580 generic.go:356] "Generic (PLEG): container finished" podID="35d04c92-1a0e-42ee-a63e-91dc5e35f966" containerID="0f31cd94b5bc19b3072ec4829911473fa9bd86bd5dddf6118833b71f0133d934" exitCode=0 Jul 06 17:39:08.075318 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:08.075180 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerDied","Data":"0f31cd94b5bc19b3072ec4829911473fa9bd86bd5dddf6118833b71f0133d934"} Jul 06 17:39:09.080486 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:09.080455 2580 generic.go:356] "Generic (PLEG): container finished" podID="35d04c92-1a0e-42ee-a63e-91dc5e35f966" containerID="3e084a668ffe51d552a974b104410e8288c135292c47b7d4823914aa1d0451a0" exitCode=0 Jul 06 17:39:09.080868 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:09.080499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerDied","Data":"3e084a668ffe51d552a974b104410e8288c135292c47b7d4823914aa1d0451a0"} Jul 06 17:39:10.072458 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:10.072421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:10.072633 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:10.072469 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:10.072633 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:10.072572 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:39:10.072633 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:10.072578 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:39:10.072633 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:10.072632 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:39:18.072612448 +0000 UTC m=+51.822980003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:39:10.072770 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:10.072642 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:39:10.072770 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:10.072681 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:39:18.072670098 +0000 UTC m=+51.823037661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:39:10.085315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:10.085273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" event={"ID":"35d04c92-1a0e-42ee-a63e-91dc5e35f966","Type":"ContainerStarted","Data":"aba34f5f9777c8d2ece72741bfffbb6616f1d74971e890a791d8309a94a7034a"} Jul 06 17:39:10.110743 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:10.109134 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wm6tm" podStartSLOduration=7.63458683 podStartE2EDuration="43.10909975s" totalImagesPullingTime="35.47451292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:39:10.106286133 +0000 UTC m=+43.856653709" watchObservedRunningTime="2026-07-06 17:39:10.10909975 +0000 UTC m=+43.859467326" Jul 06 17:39:10.226885 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:10.226852 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:10.285312 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:10.285262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:10.858131 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:10.858101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:12.673030 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:12.672995 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:12.773344 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:12.773313 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" limit=1024 Jul 06 17:39:18.128381 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:18.128347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:18.128381 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:18.128394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:18.128806 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:18.128517 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:39:18.128806 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:18.128534 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:39:18.128806 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:18.128581 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:39:34.128567538 +0000 UTC m=+67.878935106 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:39:18.128806 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:18.128516 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:39:18.128806 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:18.128654 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:39:34.128638178 +0000 UTC m=+67.879005739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:39:29.064302 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:29.064260 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w5lpr" Jul 06 17:39:32.431907 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.431854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:39:32.434923 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.434905 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 17:39:32.442317 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:32.442278 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 17:39:32.442371 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:32.442361 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs podName:d2e94037-82c6-43ee-bc44-d391a411b64d nodeName:}" failed. No retries permitted until 2026-07-06 17:40:36.44234202 +0000 UTC m=+130.192709574 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs") pod "network-metrics-daemon-nch8h" (UID: "d2e94037-82c6-43ee-bc44-d391a411b64d") : secret "metrics-daemon-secret" not found Jul 06 17:39:32.532443 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.532413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:39:32.532510 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.532492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:39:32.535630 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.535605 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 17:39:32.535751 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.535659 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 17:39:32.545418 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.545396 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 17:39:32.546766 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.546737 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/085b4edd-6175-4c47-b96f-f079c9da977b-original-pull-secret\") pod \"global-pull-secret-syncer-64lqb\" (UID: \"085b4edd-6175-4c47-b96f-f079c9da977b\") " pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:39:32.555791 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.555768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npth9\" (UniqueName: \"kubernetes.io/projected/1eb50574-dc4c-47cd-b1c1-7593c50ba6c9-kube-api-access-npth9\") pod \"network-check-target-qlnlr\" (UID: \"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9\") " pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:39:32.780782 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.780751 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64lqb" Jul 06 17:39:32.798440 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.798416 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jkf68\"" Jul 06 17:39:32.805872 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.805850 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:39:32.905067 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.905031 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-64lqb"] Jul 06 17:39:32.908487 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:39:32.908461 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod085b4edd_6175_4c47_b96f_f079c9da977b.slice/crio-34aaf8caa9f8697003e66dfc4b7e236106c7254a4e1db7285ede81e99c193509 WatchSource:0}: Error finding container 34aaf8caa9f8697003e66dfc4b7e236106c7254a4e1db7285ede81e99c193509: Status 404 returned error can't find the container with id 34aaf8caa9f8697003e66dfc4b7e236106c7254a4e1db7285ede81e99c193509 Jul 06 17:39:32.933881 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:32.933822 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qlnlr"] Jul 06 17:39:32.935799 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:39:32.935771 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb50574_dc4c_47cd_b1c1_7593c50ba6c9.slice/crio-e0d23afc99895d8f1552d8021d9430f355ae8e30bcb9fbe17c91d7fdd3fe2637 WatchSource:0}: Error finding container e0d23afc99895d8f1552d8021d9430f355ae8e30bcb9fbe17c91d7fdd3fe2637: Status 404 returned error can't find the container with id e0d23afc99895d8f1552d8021d9430f355ae8e30bcb9fbe17c91d7fdd3fe2637 Jul 06 17:39:33.138651 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:33.138568 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-64lqb" event={"ID":"085b4edd-6175-4c47-b96f-f079c9da977b","Type":"ContainerStarted","Data":"34aaf8caa9f8697003e66dfc4b7e236106c7254a4e1db7285ede81e99c193509"} Jul 06 17:39:33.139595 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:33.139567 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qlnlr" event={"ID":"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9","Type":"ContainerStarted","Data":"e0d23afc99895d8f1552d8021d9430f355ae8e30bcb9fbe17c91d7fdd3fe2637"} Jul 06 17:39:34.143957 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:34.143919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:39:34.144468 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:34.143967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:39:34.144468 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:34.144125 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:39:34.144468 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:34.144139 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:39:34.144468 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:34.144269 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:40:06.144247113 +0000 UTC m=+99.894614668 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:39:34.144468 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:34.144370 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:39:34.144468 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:39:34.144448 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:40:06.144430004 +0000 UTC m=+99.894797572 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:39:39.156671 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:39.156625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-64lqb" event={"ID":"085b4edd-6175-4c47-b96f-f079c9da977b","Type":"ContainerStarted","Data":"b126bfda4f78aac848e8f0f87f7132a9a0cf7749cbf723d4c7236a8f4ad429be"} Jul 06 17:39:39.158016 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:39.157993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qlnlr" event={"ID":"1eb50574-dc4c-47cd-b1c1-7593c50ba6c9","Type":"ContainerStarted","Data":"2129963367d2b329886cce1676dda148ec611694af4b98264bcc7962aa1ca0dc"} Jul 06 17:39:39.158142 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:39.158127 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:39:39.172011 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:39.171961 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-64lqb" podStartSLOduration=66.629871543 podStartE2EDuration="1m12.171946995s" totalImagesPullingTime="5.542075452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:39:39.171788637 +0000 UTC m=+72.922156226" watchObservedRunningTime="2026-07-06 17:39:39.171946995 +0000 UTC m=+72.922314572" Jul 06 17:39:39.186696 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:39:39.186651 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qlnlr" podStartSLOduration=66.974052396 podStartE2EDuration="1m12.186636424s" totalImagesPullingTime="5.212584028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:39:39.185545996 +0000 UTC m=+72.935913572" watchObservedRunningTime="2026-07-06 17:39:39.186636424 +0000 UTC m=+72.937003978" Jul 06 17:40:04.790807 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.790775 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vtbpl"] Jul 06 17:40:04.795118 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.795099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.798151 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.798131 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 17:40:04.798261 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.798178 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 17:40:04.798261 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.798178 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-chbhn\"" Jul 06 17:40:04.799600 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.799572 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 17:40:04.799600 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.799587 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 17:40:04.819361 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.819328 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vtbpl"] Jul 06 17:40:04.870425 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.870391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.870618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.870450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-crio-socket\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.870618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.870512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vstw\" (UniqueName: \"kubernetes.io/projected/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-api-access-2vstw\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.870618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.870557 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-data-volume\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.870618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.870575 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.971830 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.971783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.971986 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.971849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-crio-socket\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.971986 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.971866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vstw\" (UniqueName: \"kubernetes.io/projected/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-api-access-2vstw\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.971986 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.971892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-data-volume\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.971986 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.971907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.971986 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:04.971929 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:40:04.972170 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:04.972016 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:05.471998118 +0000 UTC m=+99.222365673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : configmap "kube-rbac-proxy" not found Jul 06 17:40:04.972170 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:04.972031 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:40:04.972170 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:04.972089 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:05.472074616 +0000 UTC m=+99.222442170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : secret "insights-runtime-extractor-tls" not found Jul 06 17:40:04.972170 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.971928 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-crio-socket\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.972322 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.972270 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-data-volume\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:04.980084 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:04.980050 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vstw\" (UniqueName: \"kubernetes.io/projected/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-api-access-2vstw\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:05.475350 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:05.475281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:05.475541 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:05.475384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:05.475541 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:05.475408 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:40:05.475541 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:05.475477 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:06.47546054 +0000 UTC m=+100.225828097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : configmap "kube-rbac-proxy" not found Jul 06 17:40:05.475541 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:05.475501 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:40:05.475701 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:05.475556 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:06.4755451 +0000 UTC m=+100.225912654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : secret "insights-runtime-extractor-tls" not found Jul 06 17:40:06.181515 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.181463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:40:06.181892 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.181524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") pod \"image-registry-65dc7bfff9-glg9j\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:40:06.181892 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.181612 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 17:40:06.181892 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.181618 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:40:06.181892 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.181695 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls podName:cc27b535-341b-46dd-9be8-7ce1b75e6e7f nodeName:}" failed. No retries permitted until 2026-07-06 17:41:10.181678744 +0000 UTC m=+163.932046303 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls") pod "dns-default-lkl8n" (UID: "cc27b535-341b-46dd-9be8-7ce1b75e6e7f") : secret "dns-default-metrics-tls" not found Jul 06 17:40:06.181892 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.181622 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65dc7bfff9-glg9j: secret "image-registry-tls" not found Jul 06 17:40:06.181892 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.181748 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls podName:2884e0ee-2837-48cf-abfe-5e98ff81e79a nodeName:}" failed. No retries permitted until 2026-07-06 17:41:10.181733744 +0000 UTC m=+163.932101308 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls") pod "image-registry-65dc7bfff9-glg9j" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a") : secret "image-registry-tls" not found Jul 06 17:40:06.482715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.482681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:06.482872 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.482757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:06.482872 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.482835 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:40:06.482872 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.482840 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:40:06.482967 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.482888 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:08.482874494 +0000 UTC m=+102.233242048 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : configmap "kube-rbac-proxy" not found Jul 06 17:40:06.482967 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:06.482902 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:08.482895497 +0000 UTC m=+102.233263050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : secret "insights-runtime-extractor-tls" not found Jul 06 17:40:06.593494 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.593469 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d5x7w_d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931/dns-node-resolver/0.log" Jul 06 17:40:06.792861 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.792785 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c"] Jul 06 17:40:06.795724 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.795710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" Jul 06 17:40:06.798443 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.798416 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:40:06.799684 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.799663 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tbjpf\"" Jul 06 17:40:06.799805 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.799694 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 17:40:06.804228 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.804206 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c"] Jul 06 17:40:06.885978 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.885940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djfhm\" (UniqueName: \"kubernetes.io/projected/c72fbb11-2094-4b88-b34c-16075f6be7fc-kube-api-access-djfhm\") pod \"volume-data-source-validator-5d456fdbcd-bbp2c\" (UID: \"c72fbb11-2094-4b88-b34c-16075f6be7fc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" Jul 06 17:40:06.986902 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.986854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djfhm\" (UniqueName: \"kubernetes.io/projected/c72fbb11-2094-4b88-b34c-16075f6be7fc-kube-api-access-djfhm\") pod \"volume-data-source-validator-5d456fdbcd-bbp2c\" (UID: \"c72fbb11-2094-4b88-b34c-16075f6be7fc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" Jul 06 17:40:06.996088 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:06.996053 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djfhm\" (UniqueName: \"kubernetes.io/projected/c72fbb11-2094-4b88-b34c-16075f6be7fc-kube-api-access-djfhm\") pod \"volume-data-source-validator-5d456fdbcd-bbp2c\" (UID: \"c72fbb11-2094-4b88-b34c-16075f6be7fc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" Jul 06 17:40:07.105233 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:07.105130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" Jul 06 17:40:07.228213 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:07.228190 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c"] Jul 06 17:40:07.230744 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:07.230713 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72fbb11_2094_4b88_b34c_16075f6be7fc.slice/crio-3020f4fb81078359c21e0b135a6803c75f35281f75242c86d1e78db5e8a0aba4 WatchSource:0}: Error finding container 3020f4fb81078359c21e0b135a6803c75f35281f75242c86d1e78db5e8a0aba4: Status 404 returned error can't find the container with id 3020f4fb81078359c21e0b135a6803c75f35281f75242c86d1e78db5e8a0aba4 Jul 06 17:40:07.793613 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:07.793588 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jlngt_e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3/node-ca/0.log" Jul 06 17:40:08.226363 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:08.226330 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" event={"ID":"c72fbb11-2094-4b88-b34c-16075f6be7fc","Type":"ContainerStarted","Data":"3020f4fb81078359c21e0b135a6803c75f35281f75242c86d1e78db5e8a0aba4"} Jul 06 17:40:08.392936 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:08.392907 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jvl6d_465e675d-31d0-4272-b140-23a5e780df74/serve-healthcheck-canary/0.log" Jul 06 17:40:08.499765 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:08.499657 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:08.499940 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:08.499810 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:40:08.499940 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:08.499837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:08.499940 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:08.499909 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:12.49988795 +0000 UTC m=+106.250255524 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : configmap "kube-rbac-proxy" not found Jul 06 17:40:08.500063 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:08.499958 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:40:08.500063 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:08.499994 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:12.499984514 +0000 UTC m=+106.250352069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : secret "insights-runtime-extractor-tls" not found Jul 06 17:40:09.229942 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:09.229908 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" event={"ID":"c72fbb11-2094-4b88-b34c-16075f6be7fc","Type":"ContainerStarted","Data":"6eac278791e05759edb0d3aab625c2b81ffd2db2b04e160e25586a339103f53b"} Jul 06 17:40:09.245738 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:09.245686 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-bbp2c" podStartSLOduration=1.89520725 podStartE2EDuration="3.245672581s" totalImagesPullingTime="1.350465331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:09.244922077 +0000 UTC m=+102.995289682" watchObservedRunningTime="2026-07-06 17:40:09.245672581 +0000 UTC m=+102.996040158" Jul 06 17:40:10.162851 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:10.162815 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qlnlr" Jul 06 17:40:12.529456 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:12.529417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:12.529925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:12.529473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:12.529925 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:12.529542 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:40:12.529925 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:12.529610 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:20.529593863 +0000 UTC m=+114.279961417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : configmap "kube-rbac-proxy" not found Jul 06 17:40:12.529925 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:12.529556 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:40:12.529925 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:12.529669 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:20.52965393 +0000 UTC m=+114.280021483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : secret "insights-runtime-extractor-tls" not found Jul 06 17:40:14.384765 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.384733 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7"] Jul 06 17:40:14.387224 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.387200 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" Jul 06 17:40:14.389892 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.389873 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qwktl\"" Jul 06 17:40:14.396770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.396748 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7"] Jul 06 17:40:14.443220 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.443175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmnhz\" (UniqueName: \"kubernetes.io/projected/7bd16e92-1d3b-4140-bf42-433bb0dd1c66-kube-api-access-tmnhz\") pod \"network-check-source-989f8b8dc-zcmj7\" (UID: \"7bd16e92-1d3b-4140-bf42-433bb0dd1c66\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" Jul 06 17:40:14.543659 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.543623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmnhz\" (UniqueName: \"kubernetes.io/projected/7bd16e92-1d3b-4140-bf42-433bb0dd1c66-kube-api-access-tmnhz\") pod \"network-check-source-989f8b8dc-zcmj7\" (UID: \"7bd16e92-1d3b-4140-bf42-433bb0dd1c66\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" Jul 06 17:40:14.554970 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.554925 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmnhz\" (UniqueName: \"kubernetes.io/projected/7bd16e92-1d3b-4140-bf42-433bb0dd1c66-kube-api-access-tmnhz\") pod \"network-check-source-989f8b8dc-zcmj7\" (UID: \"7bd16e92-1d3b-4140-bf42-433bb0dd1c66\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" Jul 06 17:40:14.695521 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.695424 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" Jul 06 17:40:14.818936 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:14.818903 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7"] Jul 06 17:40:14.821910 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:14.821878 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd16e92_1d3b_4140_bf42_433bb0dd1c66.slice/crio-91ee390aac4fb9e0bb9706cd04ff6c09d22166abf0b8b0d2d72414526cb9b514 WatchSource:0}: Error finding container 91ee390aac4fb9e0bb9706cd04ff6c09d22166abf0b8b0d2d72414526cb9b514: Status 404 returned error can't find the container with id 91ee390aac4fb9e0bb9706cd04ff6c09d22166abf0b8b0d2d72414526cb9b514 Jul 06 17:40:15.247522 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:15.247485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" event={"ID":"7bd16e92-1d3b-4140-bf42-433bb0dd1c66","Type":"ContainerStarted","Data":"3ba60c785336033d79229f08bd8be9da3fd60648797170affdeeb404adc03ef3"} Jul 06 17:40:15.247740 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:15.247532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" event={"ID":"7bd16e92-1d3b-4140-bf42-433bb0dd1c66","Type":"ContainerStarted","Data":"91ee390aac4fb9e0bb9706cd04ff6c09d22166abf0b8b0d2d72414526cb9b514"} Jul 06 17:40:15.265602 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:15.265551 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zcmj7" podStartSLOduration=1.2655308889999999 podStartE2EDuration="1.265530889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:15.263973505 +0000 UTC m=+109.014341081" watchObservedRunningTime="2026-07-06 17:40:15.265530889 +0000 UTC m=+109.015898465" Jul 06 17:40:17.274505 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.274460 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xpplg"] Jul 06 17:40:17.277144 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.277122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.280268 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.280248 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 17:40:17.281686 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.281660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b5nkn\"" Jul 06 17:40:17.281791 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.281709 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 17:40:17.281791 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.281738 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 17:40:17.281791 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.281740 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 17:40:17.286708 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.286688 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xpplg"] Jul 06 17:40:17.365308 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.365261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5q4x\" (UniqueName: \"kubernetes.io/projected/802e331a-8389-4504-abbd-86feab6c374a-kube-api-access-h5q4x\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.365490 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.365328 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/802e331a-8389-4504-abbd-86feab6c374a-signing-cabundle\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.365490 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.365447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/802e331a-8389-4504-abbd-86feab6c374a-signing-key\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.466704 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.466659 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/802e331a-8389-4504-abbd-86feab6c374a-signing-key\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.466799 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.466727 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5q4x\" (UniqueName: \"kubernetes.io/projected/802e331a-8389-4504-abbd-86feab6c374a-kube-api-access-h5q4x\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.466799 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.466751 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/802e331a-8389-4504-abbd-86feab6c374a-signing-cabundle\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.467438 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.467409 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/802e331a-8389-4504-abbd-86feab6c374a-signing-cabundle\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.469026 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.469004 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/802e331a-8389-4504-abbd-86feab6c374a-signing-key\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.474760 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.474736 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5q4x\" (UniqueName: \"kubernetes.io/projected/802e331a-8389-4504-abbd-86feab6c374a-kube-api-access-h5q4x\") pod \"service-ca-758f599c5-xpplg\" (UID: \"802e331a-8389-4504-abbd-86feab6c374a\") " pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.586172 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.586074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xpplg" Jul 06 17:40:17.703224 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:17.703192 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xpplg"] Jul 06 17:40:17.706136 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:17.706109 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802e331a_8389_4504_abbd_86feab6c374a.slice/crio-ed9ff53a63fe84c6fcd167eb82488392c193fdac3e458d43bd46df01ba12e0b8 WatchSource:0}: Error finding container ed9ff53a63fe84c6fcd167eb82488392c193fdac3e458d43bd46df01ba12e0b8: Status 404 returned error can't find the container with id ed9ff53a63fe84c6fcd167eb82488392c193fdac3e458d43bd46df01ba12e0b8 Jul 06 17:40:18.258610 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:18.258577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xpplg" event={"ID":"802e331a-8389-4504-abbd-86feab6c374a","Type":"ContainerStarted","Data":"ed9ff53a63fe84c6fcd167eb82488392c193fdac3e458d43bd46df01ba12e0b8"} Jul 06 17:40:20.266398 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:20.266361 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xpplg" event={"ID":"802e331a-8389-4504-abbd-86feab6c374a","Type":"ContainerStarted","Data":"1004df2a3dcb7bae8fcc906fa4e068f0ade97f70ec602f189cb94c024353fbab"} Jul 06 17:40:20.293943 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:20.293884 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xpplg" podStartSLOduration=1.2263175259999999 podStartE2EDuration="3.293867787s" totalImagesPullingTime="2.067550261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:20.29349804 +0000 UTC m=+114.043865618" watchObservedRunningTime="2026-07-06 17:40:20.293867787 +0000 UTC m=+114.044235362" Jul 06 17:40:20.591684 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:20.591579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:20.591684 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:20.591646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:20.591928 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:20.591832 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:40:20.591928 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:20.591918 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls podName:d311746f-1f0c-4269-a8dc-ab9b5900a8c9 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:36.591895565 +0000 UTC m=+130.342263133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vtbpl" (UID: "d311746f-1f0c-4269-a8dc-ab9b5900a8c9") : secret "insights-runtime-extractor-tls" not found Jul 06 17:40:20.592268 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:20.592243 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:36.516409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.516373 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:40:36.518697 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.518671 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2e94037-82c6-43ee-bc44-d391a411b64d-metrics-certs\") pod \"network-metrics-daemon-nch8h\" (UID: \"d2e94037-82c6-43ee-bc44-d391a411b64d\") " pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:40:36.617488 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.617455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:36.619961 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.619934 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d311746f-1f0c-4269-a8dc-ab9b5900a8c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vtbpl\" (UID: \"d311746f-1f0c-4269-a8dc-ab9b5900a8c9\") " pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:36.643633 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.643600 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76"] Jul 06 17:40:36.647770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.647752 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.650585 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.650559 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 17:40:36.652119 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.652095 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 17:40:36.652228 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.652206 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 17:40:36.652228 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.652216 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 17:40:36.660662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.660633 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76"] Jul 06 17:40:36.691779 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.691748 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wwzt2\"" Jul 06 17:40:36.699011 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.698989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nch8h" Jul 06 17:40:36.735689 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.735660 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89"] Jul 06 17:40:36.738938 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.738913 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" Jul 06 17:40:36.743884 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.743860 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 17:40:36.744173 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.744152 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-glvz2\"" Jul 06 17:40:36.752999 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.752967 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89"] Jul 06 17:40:36.818905 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.818875 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hfwd\" (UniqueName: \"kubernetes.io/projected/33c7ab17-09d5-4696-8072-a7f77e130f0e-kube-api-access-8hfwd\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.819068 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.818916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33c7ab17-09d5-4696-8072-a7f77e130f0e-tmp\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.819068 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.818937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/33c7ab17-09d5-4696-8072-a7f77e130f0e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.831755 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.831712 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nch8h"] Jul 06 17:40:36.834266 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:36.834235 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e94037_82c6_43ee_bc44_d391a411b64d.slice/crio-d905e17d1a171e5087c8930536139964ea62acf115cd8a052b3726d7f6ac37ee WatchSource:0}: Error finding container d905e17d1a171e5087c8930536139964ea62acf115cd8a052b3726d7f6ac37ee: Status 404 returned error can't find the container with id d905e17d1a171e5087c8930536139964ea62acf115cd8a052b3726d7f6ac37ee Jul 06 17:40:36.907630 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.907602 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-chbhn\"" Jul 06 17:40:36.915000 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.914983 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vtbpl" Jul 06 17:40:36.919325 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.919282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33c7ab17-09d5-4696-8072-a7f77e130f0e-tmp\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.919447 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.919330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/33c7ab17-09d5-4696-8072-a7f77e130f0e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.919447 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.919397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-d8p89\" (UID: \"f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" Jul 06 17:40:36.919560 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.919447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hfwd\" (UniqueName: \"kubernetes.io/projected/33c7ab17-09d5-4696-8072-a7f77e130f0e-kube-api-access-8hfwd\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.919685 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.919660 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33c7ab17-09d5-4696-8072-a7f77e130f0e-tmp\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.921857 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.921834 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/33c7ab17-09d5-4696-8072-a7f77e130f0e-klusterlet-config\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.931376 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.931347 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hfwd\" (UniqueName: \"kubernetes.io/projected/33c7ab17-09d5-4696-8072-a7f77e130f0e-kube-api-access-8hfwd\") pod \"klusterlet-addon-workmgr-7c48458575-q5k76\" (UID: \"33c7ab17-09d5-4696-8072-a7f77e130f0e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:36.957696 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:36.957662 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:37.021154 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.021107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-d8p89\" (UID: \"f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" Jul 06 17:40:37.024185 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.024163 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-d8p89\" (UID: \"f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" Jul 06 17:40:37.047925 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.047826 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vtbpl"] Jul 06 17:40:37.050158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.049953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" Jul 06 17:40:37.050267 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:37.050163 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd311746f_1f0c_4269_a8dc_ab9b5900a8c9.slice/crio-04e553b240ece6fb12fa9d79a6f0d843f1cc724be5199a75e4ac075414f998df WatchSource:0}: Error finding container 04e553b240ece6fb12fa9d79a6f0d843f1cc724be5199a75e4ac075414f998df: Status 404 returned error can't find the container with id 04e553b240ece6fb12fa9d79a6f0d843f1cc724be5199a75e4ac075414f998df Jul 06 17:40:37.100212 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.100177 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76"] Jul 06 17:40:37.103061 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:37.103027 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c7ab17_09d5_4696_8072_a7f77e130f0e.slice/crio-4e6cb4dc9c3f7b3f11d8a48a2a4a7c124c931a636861104bc9824c581ef48580 WatchSource:0}: Error finding container 4e6cb4dc9c3f7b3f11d8a48a2a4a7c124c931a636861104bc9824c581ef48580: Status 404 returned error can't find the container with id 4e6cb4dc9c3f7b3f11d8a48a2a4a7c124c931a636861104bc9824c581ef48580 Jul 06 17:40:37.183968 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.183938 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89"] Jul 06 17:40:37.186110 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:37.186080 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1879a5b_52b9_46d8_abd5_cf2ddd6ffac3.slice/crio-c96bc0596955da04433f5cff1f124ae753ca7bd5ad5c53d93fcb27debbd6336f WatchSource:0}: Error finding container c96bc0596955da04433f5cff1f124ae753ca7bd5ad5c53d93fcb27debbd6336f: Status 404 returned error can't find the container with id c96bc0596955da04433f5cff1f124ae753ca7bd5ad5c53d93fcb27debbd6336f Jul 06 17:40:37.323197 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.323102 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nch8h" event={"ID":"d2e94037-82c6-43ee-bc44-d391a411b64d","Type":"ContainerStarted","Data":"d905e17d1a171e5087c8930536139964ea62acf115cd8a052b3726d7f6ac37ee"} Jul 06 17:40:37.324583 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.324547 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtbpl" event={"ID":"d311746f-1f0c-4269-a8dc-ab9b5900a8c9","Type":"ContainerStarted","Data":"04e553b240ece6fb12fa9d79a6f0d843f1cc724be5199a75e4ac075414f998df"} Jul 06 17:40:37.325890 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.325860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" event={"ID":"f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3","Type":"ContainerStarted","Data":"c96bc0596955da04433f5cff1f124ae753ca7bd5ad5c53d93fcb27debbd6336f"} Jul 06 17:40:37.327029 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:37.327008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" event={"ID":"33c7ab17-09d5-4696-8072-a7f77e130f0e","Type":"ContainerStarted","Data":"4e6cb4dc9c3f7b3f11d8a48a2a4a7c124c931a636861104bc9824c581ef48580"} Jul 06 17:40:39.339479 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.339395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" event={"ID":"f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3","Type":"ContainerStarted","Data":"b47919d9734245c79515716c92ed6c3f29daf058b653ccfcbd890543af5c05f6"} Jul 06 17:40:39.339967 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.339674 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" Jul 06 17:40:39.342057 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.342028 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nch8h" event={"ID":"d2e94037-82c6-43ee-bc44-d391a411b64d","Type":"ContainerStarted","Data":"79d0df352c19a21ee37bc11298b0d2b6c43e4dfabde33432c11772b23e4b9999"} Jul 06 17:40:39.342190 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.342083 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nch8h" event={"ID":"d2e94037-82c6-43ee-bc44-d391a411b64d","Type":"ContainerStarted","Data":"cad8a09324647b244bb699edf76762fee61a68da336bbbf1b43832e09d232d2d"} Jul 06 17:40:39.345954 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.345931 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" Jul 06 17:40:39.360696 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.360634 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-d8p89" podStartSLOduration=1.737533457 podStartE2EDuration="3.360613421s" totalImagesPullingTime="1.623079964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:39.359426493 +0000 UTC m=+133.109794070" watchObservedRunningTime="2026-07-06 17:40:39.360613421 +0000 UTC m=+133.110981001" Jul 06 17:40:39.395990 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.395701 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nch8h" podStartSLOduration=130.42476451 podStartE2EDuration="2m12.395681404s" totalImagesPullingTime="1.970916894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:39.395486231 +0000 UTC m=+133.145853826" watchObservedRunningTime="2026-07-06 17:40:39.395681404 +0000 UTC m=+133.146048980" Jul 06 17:40:39.755955 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.755919 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf"] Jul 06 17:40:39.760389 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.760364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.763770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.763421 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 17:40:39.763770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.763480 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 17:40:39.763770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.763603 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 17:40:39.763770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.763636 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 17:40:39.763770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.763651 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hb697\"" Jul 06 17:40:39.764127 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.763962 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 17:40:39.769520 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.769458 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf"] Jul 06 17:40:39.848697 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.848442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.848697 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.848560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.848697 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.848589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p57lv\" (UniqueName: \"kubernetes.io/projected/66931001-3fdb-4df8-bc40-a88daa32bdac-kube-api-access-p57lv\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.848697 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.848619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66931001-3fdb-4df8-bc40-a88daa32bdac-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.949331 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.949266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.949543 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.949383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.949543 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.949446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p57lv\" (UniqueName: \"kubernetes.io/projected/66931001-3fdb-4df8-bc40-a88daa32bdac-kube-api-access-p57lv\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.949543 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.949480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66931001-3fdb-4df8-bc40-a88daa32bdac-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.949885 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:39.949702 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 17:40:39.949885 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:39.949776 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls podName:66931001-3fdb-4df8-bc40-a88daa32bdac nodeName:}" failed. No retries permitted until 2026-07-06 17:40:40.449757685 +0000 UTC m=+134.200125257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-ls7xf" (UID: "66931001-3fdb-4df8-bc40-a88daa32bdac") : secret "prometheus-operator-tls" not found Jul 06 17:40:39.950220 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.950197 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66931001-3fdb-4df8-bc40-a88daa32bdac-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.953102 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.953061 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:39.962073 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:39.962048 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p57lv\" (UniqueName: \"kubernetes.io/projected/66931001-3fdb-4df8-bc40-a88daa32bdac-kube-api-access-p57lv\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:40.453115 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:40.453076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:40.453609 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:40.453247 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 17:40:40.453609 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:40.453350 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls podName:66931001-3fdb-4df8-bc40-a88daa32bdac nodeName:}" failed. No retries permitted until 2026-07-06 17:40:41.453327699 +0000 UTC m=+135.203695275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-ls7xf" (UID: "66931001-3fdb-4df8-bc40-a88daa32bdac") : secret "prometheus-operator-tls" not found Jul 06 17:40:41.462516 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:41.462473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:41.465429 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:41.465393 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/66931001-3fdb-4df8-bc40-a88daa32bdac-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-ls7xf\" (UID: \"66931001-3fdb-4df8-bc40-a88daa32bdac\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:41.573382 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:41.573347 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" Jul 06 17:40:43.406843 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:43.406774 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf"] Jul 06 17:40:43.408918 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:43.408896 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66931001_3fdb_4df8_bc40_a88daa32bdac.slice/crio-964f9e93553c74acf52da37b73b0d55ae69a921c2e5573475e458ae18b471123 WatchSource:0}: Error finding container 964f9e93553c74acf52da37b73b0d55ae69a921c2e5573475e458ae18b471123: Status 404 returned error can't find the container with id 964f9e93553c74acf52da37b73b0d55ae69a921c2e5573475e458ae18b471123 Jul 06 17:40:44.363209 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:44.363168 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" event={"ID":"33c7ab17-09d5-4696-8072-a7f77e130f0e","Type":"ContainerStarted","Data":"1c82d1542fcaf6143352a3c8b0a57141c2aeb878b499d6eca6611fce06bda390"} Jul 06 17:40:44.363439 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:44.363412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:44.364605 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:44.364576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" event={"ID":"66931001-3fdb-4df8-bc40-a88daa32bdac","Type":"ContainerStarted","Data":"964f9e93553c74acf52da37b73b0d55ae69a921c2e5573475e458ae18b471123"} Jul 06 17:40:44.365088 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:44.365072 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" Jul 06 17:40:44.366754 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:44.366728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtbpl" event={"ID":"d311746f-1f0c-4269-a8dc-ab9b5900a8c9","Type":"ContainerStarted","Data":"e3640442b7e2fb058d227cbe485fae47f1d891b3efc230e0adbbc3dbc747d8d5"} Jul 06 17:40:44.366861 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:44.366783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtbpl" event={"ID":"d311746f-1f0c-4269-a8dc-ab9b5900a8c9","Type":"ContainerStarted","Data":"04df2365e9aeed6835bd28416008fbb43a4d52f59f0453f0bf885c390b1b6d29"} Jul 06 17:40:44.380344 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:44.380271 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7c48458575-q5k76" podStartSLOduration=2.193505952 podStartE2EDuration="8.38025853s" totalImagesPullingTime="6.186752578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:44.379736773 +0000 UTC m=+138.130104350" watchObservedRunningTime="2026-07-06 17:40:44.38025853 +0000 UTC m=+138.130626107" Jul 06 17:40:45.371869 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:45.371817 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" event={"ID":"66931001-3fdb-4df8-bc40-a88daa32bdac","Type":"ContainerStarted","Data":"b6d58ad5c265bf19a98aacfbaf1874d7abb60d08da0c179a2e701ef3ec2900b6"} Jul 06 17:40:45.372346 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:45.371917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" event={"ID":"66931001-3fdb-4df8-bc40-a88daa32bdac","Type":"ContainerStarted","Data":"063669f610aa59a1b31b4b8a97365b2e06178f6af6542c827a6467d5b896a258"} Jul 06 17:40:45.389835 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:45.389777 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-ls7xf" podStartSLOduration=5.154813086 podStartE2EDuration="6.389757181s" totalImagesPullingTime="1.234944095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:45.389014576 +0000 UTC m=+139.139382152" watchObservedRunningTime="2026-07-06 17:40:45.389757181 +0000 UTC m=+139.140124756" Jul 06 17:40:47.085501 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.085412 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h"] Jul 06 17:40:47.088679 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.088662 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.091723 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.091702 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 17:40:47.091872 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.091783 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mwxhx\"" Jul 06 17:40:47.091872 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.091816 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 17:40:47.098880 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.098850 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h"] Jul 06 17:40:47.108544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.108514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.108646 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.108574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7040cc35-7fab-450e-b259-d22694151513-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.108693 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.108654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvzdw\" (UniqueName: \"kubernetes.io/projected/7040cc35-7fab-450e-b259-d22694151513-kube-api-access-nvzdw\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.108737 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.108724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.135058 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.135025 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4rjm5"] Jul 06 17:40:47.138262 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.138242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.141227 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.141204 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 17:40:47.141380 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.141233 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 17:40:47.141380 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.141274 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 17:40:47.141380 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.141325 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8fxcc\"" Jul 06 17:40:47.209191 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.209421 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-accelerators-collector-config\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209421 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-root\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209421 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.209421 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:47.209335 2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 17:40:47.209421 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-textfile\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209421 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209421 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:47.209418 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-tls podName:7040cc35-7fab-450e-b259-d22694151513 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:47.709397746 +0000 UTC m=+141.459765311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-6ll6h" (UID: "7040cc35-7fab-450e-b259-d22694151513") : secret "openshift-state-metrics-tls" not found Jul 06 17:40:47.209790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-wtmp\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32d03f1e-5b97-42c3-bf29-02159bc2955d-metrics-client-ca\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7040cc35-7fab-450e-b259-d22694151513-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.209790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvzdw\" (UniqueName: \"kubernetes.io/projected/7040cc35-7fab-450e-b259-d22694151513-kube-api-access-nvzdw\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.209790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209640 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-sys\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7m9h\" (UniqueName: \"kubernetes.io/projected/32d03f1e-5b97-42c3-bf29-02159bc2955d-kube-api-access-j7m9h\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.209790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.209700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-tls\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.210251 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.210229 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7040cc35-7fab-450e-b259-d22694151513-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.211786 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.211768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.220274 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.220245 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvzdw\" (UniqueName: \"kubernetes.io/projected/7040cc35-7fab-450e-b259-d22694151513-kube-api-access-nvzdw\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.310276 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-textfile\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310276 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-wtmp\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32d03f1e-5b97-42c3-bf29-02159bc2955d-metrics-client-ca\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-sys\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7m9h\" (UniqueName: \"kubernetes.io/projected/32d03f1e-5b97-42c3-bf29-02159bc2955d-kube-api-access-j7m9h\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310449 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-tls\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-accelerators-collector-config\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-root\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310876 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310536 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-wtmp\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310876 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310586 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-root\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310876 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310681 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-textfile\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.310876 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310852 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32d03f1e-5b97-42c3-bf29-02159bc2955d-sys\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.311064 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.310995 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32d03f1e-5b97-42c3-bf29-02159bc2955d-metrics-client-ca\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.311162 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.311142 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-accelerators-collector-config\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.312798 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.312776 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-tls\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.312911 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.312896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32d03f1e-5b97-42c3-bf29-02159bc2955d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.318582 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.318562 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7m9h\" (UniqueName: \"kubernetes.io/projected/32d03f1e-5b97-42c3-bf29-02159bc2955d-kube-api-access-j7m9h\") pod \"node-exporter-4rjm5\" (UID: \"32d03f1e-5b97-42c3-bf29-02159bc2955d\") " pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.384806 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.384719 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vtbpl" event={"ID":"d311746f-1f0c-4269-a8dc-ab9b5900a8c9","Type":"ContainerStarted","Data":"d29a196992706278eba4e085130bbee71ab522bfaa66e3332ac577d617f9cf1e"} Jul 06 17:40:47.402005 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.401946 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vtbpl" podStartSLOduration=33.839557817 podStartE2EDuration="43.401929198s" totalImagesPullingTime="9.562371381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:47.401509742 +0000 UTC m=+141.151877318" watchObservedRunningTime="2026-07-06 17:40:47.401929198 +0000 UTC m=+141.152296775" Jul 06 17:40:47.449451 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.449420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4rjm5" Jul 06 17:40:47.461442 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:47.461396 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d03f1e_5b97_42c3_bf29_02159bc2955d.slice/crio-710caea24141b0abfd8ea8ef051b79c5b1700f566f0a2c885413b0209b450c46 WatchSource:0}: Error finding container 710caea24141b0abfd8ea8ef051b79c5b1700f566f0a2c885413b0209b450c46: Status 404 returned error can't find the container with id 710caea24141b0abfd8ea8ef051b79c5b1700f566f0a2c885413b0209b450c46 Jul 06 17:40:47.714179 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.714085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.716451 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.716422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7040cc35-7fab-450e-b259-d22694151513-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-6ll6h\" (UID: \"7040cc35-7fab-450e-b259-d22694151513\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:47.999531 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:47.999496 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" Jul 06 17:40:48.149798 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:48.149756 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h"] Jul 06 17:40:48.153616 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:48.153576 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7040cc35_7fab_450e_b259_d22694151513.slice/crio-9b0c43635f36f0e886738adafe178ccfd535f4d34d5cbc6f2f76fcc5fb1d7a42 WatchSource:0}: Error finding container 9b0c43635f36f0e886738adafe178ccfd535f4d34d5cbc6f2f76fcc5fb1d7a42: Status 404 returned error can't find the container with id 9b0c43635f36f0e886738adafe178ccfd535f4d34d5cbc6f2f76fcc5fb1d7a42 Jul 06 17:40:48.390141 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:48.390100 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" event={"ID":"7040cc35-7fab-450e-b259-d22694151513","Type":"ContainerStarted","Data":"3e71a97b8d3bf033e4ea9cab373e6034b5133961c14d9a8596efd331f2f81f91"} Jul 06 17:40:48.390339 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:48.390175 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" event={"ID":"7040cc35-7fab-450e-b259-d22694151513","Type":"ContainerStarted","Data":"e8db3497c09efcb4d811e4988b25e51b3df8c93085d7115744ced589104ab9fb"} Jul 06 17:40:48.390339 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:48.390204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" event={"ID":"7040cc35-7fab-450e-b259-d22694151513","Type":"ContainerStarted","Data":"9b0c43635f36f0e886738adafe178ccfd535f4d34d5cbc6f2f76fcc5fb1d7a42"} Jul 06 17:40:48.391565 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:48.391535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4rjm5" event={"ID":"32d03f1e-5b97-42c3-bf29-02159bc2955d","Type":"ContainerStarted","Data":"710caea24141b0abfd8ea8ef051b79c5b1700f566f0a2c885413b0209b450c46"} Jul 06 17:40:49.400096 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:49.400063 2580 generic.go:356] "Generic (PLEG): container finished" podID="32d03f1e-5b97-42c3-bf29-02159bc2955d" containerID="c5b61d88dee29fab7c4cb265b5ecafd446142add33fb26247074e271217efc97" exitCode=0 Jul 06 17:40:49.400697 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:49.400161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4rjm5" event={"ID":"32d03f1e-5b97-42c3-bf29-02159bc2955d","Type":"ContainerDied","Data":"c5b61d88dee29fab7c4cb265b5ecafd446142add33fb26247074e271217efc97"} Jul 06 17:40:50.074206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.074168 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-76f6c55d98-h2czn"] Jul 06 17:40:50.077804 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.077783 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.081206 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.081181 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 17:40:50.081538 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.081521 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 17:40:50.081792 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.081775 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2hac427lt3ap\"" Jul 06 17:40:50.081996 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.081972 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zthgt\"" Jul 06 17:40:50.082117 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.082047 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 17:40:50.082537 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.082515 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 17:40:50.082628 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.082522 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 17:40:50.095137 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.095111 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-76f6c55d98-h2czn"] Jul 06 17:40:50.135924 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.135890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.136124 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.135950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.136124 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.136024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-tls\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.136124 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.136066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptzgh\" (UniqueName: \"kubernetes.io/projected/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-kube-api-access-ptzgh\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.136124 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.136089 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.136280 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.136123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-metrics-client-ca\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.136280 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.136163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-grpc-tls\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.136280 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.136188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.236861 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.236801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237079 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.236873 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237079 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.237032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-tls\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237207 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.237081 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptzgh\" (UniqueName: \"kubernetes.io/projected/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-kube-api-access-ptzgh\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237207 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.237111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237207 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.237136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-metrics-client-ca\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237550 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.237190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-grpc-tls\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237622 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.237608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.237955 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.237908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-metrics-client-ca\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.240013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.239987 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.240542 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.240509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-grpc-tls\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.240542 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.240519 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.240706 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.240543 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-tls\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.240706 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.240602 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.240706 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.240657 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.246134 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.246111 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptzgh\" (UniqueName: \"kubernetes.io/projected/d9b57b0f-9235-4e27-b4e8-993443f6dfb2-kube-api-access-ptzgh\") pod \"thanos-querier-76f6c55d98-h2czn\" (UID: \"d9b57b0f-9235-4e27-b4e8-993443f6dfb2\") " pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.387893 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.387795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:50.406812 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.406772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" event={"ID":"7040cc35-7fab-450e-b259-d22694151513","Type":"ContainerStarted","Data":"b0af07fead73e2b1490c8581ca90ab0ee3500a8f9128e78a6006b77193f433ab"} Jul 06 17:40:50.409363 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.409331 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4rjm5" event={"ID":"32d03f1e-5b97-42c3-bf29-02159bc2955d","Type":"ContainerStarted","Data":"d10cb89848bde0d8cab44db984870964d509b5a43dcb1580084fe8cfd4c2ca11"} Jul 06 17:40:50.409486 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.409394 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4rjm5" event={"ID":"32d03f1e-5b97-42c3-bf29-02159bc2955d","Type":"ContainerStarted","Data":"658d44fe22205d0432412782355222dea3898ad4f546ebca9fc4dd5332b0512a"} Jul 06 17:40:50.426750 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.426686 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-6ll6h" podStartSLOduration=2.156602681 podStartE2EDuration="3.42666598s" totalImagesPullingTime="1.270063299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:50.424636052 +0000 UTC m=+144.175003633" watchObservedRunningTime="2026-07-06 17:40:50.42666598 +0000 UTC m=+144.177033557" Jul 06 17:40:50.443945 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.442747 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4rjm5" podStartSLOduration=2.244041188 podStartE2EDuration="3.442721718s" totalImagesPullingTime="1.133132285s" totalInitContainerRuntime="65.548245ms" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:50.441980396 +0000 UTC m=+144.192347972" watchObservedRunningTime="2026-07-06 17:40:50.442721718 +0000 UTC m=+144.193089298" Jul 06 17:40:50.528096 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:50.528061 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-76f6c55d98-h2czn"] Jul 06 17:40:50.531079 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:50.531053 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b57b0f_9235_4e27_b4e8_993443f6dfb2.slice/crio-3d63bd19ec7a6a45e7bb25c0fe9dad43c4b3e0662b47500ca5a63d8277805b81 WatchSource:0}: Error finding container 3d63bd19ec7a6a45e7bb25c0fe9dad43c4b3e0662b47500ca5a63d8277805b81: Status 404 returned error can't find the container with id 3d63bd19ec7a6a45e7bb25c0fe9dad43c4b3e0662b47500ca5a63d8277805b81 Jul 06 17:40:51.413972 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.413938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" event={"ID":"d9b57b0f-9235-4e27-b4e8-993443f6dfb2","Type":"ContainerStarted","Data":"3d63bd19ec7a6a45e7bb25c0fe9dad43c4b3e0662b47500ca5a63d8277805b81"} Jul 06 17:40:51.494853 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.494810 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6d9668478f-bs6fw"] Jul 06 17:40:51.497856 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.497835 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.501249 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.501073 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 17:40:51.502846 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.502818 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ph54t\"" Jul 06 17:40:51.502846 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.502831 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 17:40:51.503003 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.502859 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 17:40:51.503003 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.502822 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-a53e3rikfhdc3\"" Jul 06 17:40:51.503003 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.502885 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 17:40:51.509410 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.509389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d9668478f-bs6fw"] Jul 06 17:40:51.550122 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.550085 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsnqw\" (UniqueName: \"kubernetes.io/projected/80bd5c42-7663-43f2-9e10-9f2f59d630d3-kube-api-access-bsnqw\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.550350 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.550170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-secret-metrics-server-client-certs\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.550350 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.550255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80bd5c42-7663-43f2-9e10-9f2f59d630d3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.550350 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.550319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-secret-metrics-server-tls\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.550524 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.550351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/80bd5c42-7663-43f2-9e10-9f2f59d630d3-audit-log\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.550524 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.550386 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/80bd5c42-7663-43f2-9e10-9f2f59d630d3-metrics-server-audit-profiles\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.550524 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.550412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-client-ca-bundle\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.651442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80bd5c42-7663-43f2-9e10-9f2f59d630d3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.651442 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-secret-metrics-server-tls\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.651698 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/80bd5c42-7663-43f2-9e10-9f2f59d630d3-audit-log\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.651698 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/80bd5c42-7663-43f2-9e10-9f2f59d630d3-metrics-server-audit-profiles\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.651698 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-client-ca-bundle\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.651698 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsnqw\" (UniqueName: \"kubernetes.io/projected/80bd5c42-7663-43f2-9e10-9f2f59d630d3-kube-api-access-bsnqw\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.651698 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-secret-metrics-server-client-certs\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.652021 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.651991 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/80bd5c42-7663-43f2-9e10-9f2f59d630d3-audit-log\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.652247 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.652226 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80bd5c42-7663-43f2-9e10-9f2f59d630d3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.652568 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.652535 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/80bd5c42-7663-43f2-9e10-9f2f59d630d3-metrics-server-audit-profiles\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.654001 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.653982 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-secret-metrics-server-client-certs\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.654083 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.654067 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-secret-metrics-server-tls\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.654226 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.654204 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd5c42-7663-43f2-9e10-9f2f59d630d3-client-ca-bundle\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.665872 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.665819 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-dm7mj"] Jul 06 17:40:51.668927 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.668908 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsnqw\" (UniqueName: \"kubernetes.io/projected/80bd5c42-7663-43f2-9e10-9f2f59d630d3-kube-api-access-bsnqw\") pod \"metrics-server-6d9668478f-bs6fw\" (UID: \"80bd5c42-7663-43f2-9e10-9f2f59d630d3\") " pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.669979 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.669961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-dm7mj" Jul 06 17:40:51.672722 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.672706 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5qwc5\"" Jul 06 17:40:51.672974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.672940 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 17:40:51.673202 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.673189 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 17:40:51.681346 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.681327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-dm7mj"] Jul 06 17:40:51.752199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.752163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5rnc\" (UniqueName: \"kubernetes.io/projected/ba2ffe7e-2e50-42b5-8dca-7f4f2b119381-kube-api-access-q5rnc\") pod \"downloads-685bfb5f64-dm7mj\" (UID: \"ba2ffe7e-2e50-42b5-8dca-7f4f2b119381\") " pod="openshift-console/downloads-685bfb5f64-dm7mj" Jul 06 17:40:51.809374 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.809340 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:40:51.853137 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.853082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5rnc\" (UniqueName: \"kubernetes.io/projected/ba2ffe7e-2e50-42b5-8dca-7f4f2b119381-kube-api-access-q5rnc\") pod \"downloads-685bfb5f64-dm7mj\" (UID: \"ba2ffe7e-2e50-42b5-8dca-7f4f2b119381\") " pod="openshift-console/downloads-685bfb5f64-dm7mj" Jul 06 17:40:51.868822 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.868797 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5rnc\" (UniqueName: \"kubernetes.io/projected/ba2ffe7e-2e50-42b5-8dca-7f4f2b119381-kube-api-access-q5rnc\") pod \"downloads-685bfb5f64-dm7mj\" (UID: \"ba2ffe7e-2e50-42b5-8dca-7f4f2b119381\") " pod="openshift-console/downloads-685bfb5f64-dm7mj" Jul 06 17:40:51.952645 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.952616 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6d9668478f-bs6fw"] Jul 06 17:40:51.955248 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:51.955222 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80bd5c42_7663_43f2_9e10_9f2f59d630d3.slice/crio-2012dee254292b2fa1390d58766fb952d9b92bc33d8078f4a507eef18193b8e0 WatchSource:0}: Error finding container 2012dee254292b2fa1390d58766fb952d9b92bc33d8078f4a507eef18193b8e0: Status 404 returned error can't find the container with id 2012dee254292b2fa1390d58766fb952d9b92bc33d8078f4a507eef18193b8e0 Jul 06 17:40:51.982617 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:51.982579 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-dm7mj" Jul 06 17:40:52.115751 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.115679 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-dm7mj"] Jul 06 17:40:52.117833 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:52.117806 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2ffe7e_2e50_42b5_8dca_7f4f2b119381.slice/crio-3d459a2f04578b6c86545d478f9c60b42fe8e89ab8768550a7252bbc09f7e77c WatchSource:0}: Error finding container 3d459a2f04578b6c86545d478f9c60b42fe8e89ab8768550a7252bbc09f7e77c: Status 404 returned error can't find the container with id 3d459a2f04578b6c86545d478f9c60b42fe8e89ab8768550a7252bbc09f7e77c Jul 06 17:40:52.310400 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.310339 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd"] Jul 06 17:40:52.314139 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.314112 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.316943 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.316913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 17:40:52.316943 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.316913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 17:40:52.317405 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.317382 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 17:40:52.317496 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.317477 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 17:40:52.317832 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.317792 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 17:40:52.317933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.317853 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wxp75\"" Jul 06 17:40:52.323266 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.323236 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 17:40:52.327520 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.327500 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd"] Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-telemeter-client-tls\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-federate-client-tls\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-metrics-client-ca\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-secret-telemeter-client\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-serving-certs-ca-bundle\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.362597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.359485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mmfs\" (UniqueName: \"kubernetes.io/projected/18db26d8-6e39-4e47-a2d2-6c196853c659-kube-api-access-4mmfs\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.419517 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.419477 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" event={"ID":"80bd5c42-7663-43f2-9e10-9f2f59d630d3","Type":"ContainerStarted","Data":"2012dee254292b2fa1390d58766fb952d9b92bc33d8078f4a507eef18193b8e0"} Jul 06 17:40:52.420763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.420732 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-dm7mj" event={"ID":"ba2ffe7e-2e50-42b5-8dca-7f4f2b119381","Type":"ContainerStarted","Data":"3d459a2f04578b6c86545d478f9c60b42fe8e89ab8768550a7252bbc09f7e77c"} Jul 06 17:40:52.460609 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.460567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-telemeter-client-tls\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.460790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.460637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.460790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.460678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-federate-client-tls\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.460790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.460706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-metrics-client-ca\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.460790 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.460741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-secret-telemeter-client\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.461003 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.460805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-serving-certs-ca-bundle\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.461249 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.461132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.461249 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.461187 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mmfs\" (UniqueName: \"kubernetes.io/projected/18db26d8-6e39-4e47-a2d2-6c196853c659-kube-api-access-4mmfs\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.462199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.461557 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-serving-certs-ca-bundle\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.462199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.461635 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-metrics-client-ca\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.462199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.462156 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18db26d8-6e39-4e47-a2d2-6c196853c659-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.463709 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.463685 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-telemeter-client-tls\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.463945 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.463918 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.464061 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.464007 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-federate-client-tls\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.464112 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.464094 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/18db26d8-6e39-4e47-a2d2-6c196853c659-secret-telemeter-client\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.471137 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.471113 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mmfs\" (UniqueName: \"kubernetes.io/projected/18db26d8-6e39-4e47-a2d2-6c196853c659-kube-api-access-4mmfs\") pod \"telemeter-client-5d7df4f5cc-xswfd\" (UID: \"18db26d8-6e39-4e47-a2d2-6c196853c659\") " pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.625717 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.625632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" Jul 06 17:40:52.851335 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:52.851307 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd"] Jul 06 17:40:52.854524 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:52.854490 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18db26d8_6e39_4e47_a2d2_6c196853c659.slice/crio-493f8bd776429e046ad95d4f4f89be5ca68570c1fc2deb017aae2c663aa52f15 WatchSource:0}: Error finding container 493f8bd776429e046ad95d4f4f89be5ca68570c1fc2deb017aae2c663aa52f15: Status 404 returned error can't find the container with id 493f8bd776429e046ad95d4f4f89be5ca68570c1fc2deb017aae2c663aa52f15 Jul 06 17:40:53.324246 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.322209 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:40:53.337641 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.334023 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.338307 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.338611 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.338849 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.339274 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.339577 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.339756 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.339974 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.340159 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.340528 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.340722 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.340734 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.340898 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.340951 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.341137 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c93125qn5pdqi\"" Jul 06 17:40:53.341277 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.341176 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 17:40:53.345303 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.344657 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 17:40:53.345303 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.345119 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2tx77\"" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373536 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373641 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373680 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2jl6\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-kube-api-access-p2jl6\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373808 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373836 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.374271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.375199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.375199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.373951 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.375199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.374007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.375199 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.374065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.429179 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.429130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" event={"ID":"18db26d8-6e39-4e47-a2d2-6c196853c659","Type":"ContainerStarted","Data":"493f8bd776429e046ad95d4f4f89be5ca68570c1fc2deb017aae2c663aa52f15"} Jul 06 17:40:53.437253 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.437028 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" event={"ID":"d9b57b0f-9235-4e27-b4e8-993443f6dfb2","Type":"ContainerStarted","Data":"0a1accfb52745e76567c814d3661cba1a57732e15c9ca5cc171988fdbbd53692"} Jul 06 17:40:53.437253 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.437110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" event={"ID":"d9b57b0f-9235-4e27-b4e8-993443f6dfb2","Type":"ContainerStarted","Data":"05707d67949c6f2ac3e2ebecc122ef90a9e99d947a8eeab323be0379eec0fb36"} Jul 06 17:40:53.437253 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.437153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" event={"ID":"d9b57b0f-9235-4e27-b4e8-993443f6dfb2","Type":"ContainerStarted","Data":"1c10eff1ef55f81d9b2bc1645ac47fd90ddd064b6ecead6adde4971b0c2d7bdf"} Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.475937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476068 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476093 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.476639 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.479776 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.476516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.479776 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.478765 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.479776 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.478799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.479776 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.478840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2jl6\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-kube-api-access-p2jl6\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.479776 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.479190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.479776 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.479775 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.481526 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.481402 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.481910 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.481879 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.482003 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.481940 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.482572 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.482545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.482651 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.482585 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.482853 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.482828 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.482933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.482855 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.483731 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.483373 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.483731 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.483673 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.484708 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.484684 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.487872 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.487847 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.491546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.491523 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.492306 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.491803 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.492306 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.491915 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.492306 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.492250 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.492520 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.492314 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.493513 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.493472 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.495133 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.495088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2jl6\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-kube-api-access-p2jl6\") pod \"prometheus-k8s-0\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:53.655466 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:53.655381 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:40:54.620433 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:54.620393 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:40:55.075032 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:40:55.075000 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef629912_a692_4e12_a7ef_7255d1ed6c2a.slice/crio-68d7838aeadd66ec86ed6807315f777b63046802ee51d79c0cdd6b2d4a82b79e WatchSource:0}: Error finding container 68d7838aeadd66ec86ed6807315f777b63046802ee51d79c0cdd6b2d4a82b79e: Status 404 returned error can't find the container with id 68d7838aeadd66ec86ed6807315f777b63046802ee51d79c0cdd6b2d4a82b79e Jul 06 17:40:55.459189 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.459106 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" event={"ID":"80bd5c42-7663-43f2-9e10-9f2f59d630d3","Type":"ContainerStarted","Data":"86d54345cf64318512a7fae16ab8f166859e46472f96c59c5d46d54204f9e3d5"} Jul 06 17:40:55.460473 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.460443 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerStarted","Data":"68d7838aeadd66ec86ed6807315f777b63046802ee51d79c0cdd6b2d4a82b79e"} Jul 06 17:40:55.462082 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.462054 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" event={"ID":"18db26d8-6e39-4e47-a2d2-6c196853c659","Type":"ContainerStarted","Data":"f306383a9345f4bc4b1c8adccc0cc05b4625750a8dfd3315ae9d0d315a5a4e69"} Jul 06 17:40:55.465940 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.465916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" event={"ID":"d9b57b0f-9235-4e27-b4e8-993443f6dfb2","Type":"ContainerStarted","Data":"fde1e8c6e416d7be7f185b2e7156dad8c4b1839b6703d7c91e3d57abe2a6b98c"} Jul 06 17:40:55.466051 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.466013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" event={"ID":"d9b57b0f-9235-4e27-b4e8-993443f6dfb2","Type":"ContainerStarted","Data":"689bc13b233620d1fc9b010553e8660050248be712ca8fb4950a72943af05895"} Jul 06 17:40:55.466113 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.466073 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:40:55.466176 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.466115 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" event={"ID":"d9b57b0f-9235-4e27-b4e8-993443f6dfb2","Type":"ContainerStarted","Data":"33dd26d3e9d8d349c6a6875fd793dab31ee4a3288226d026f8f5a2790a75b0f5"} Jul 06 17:40:55.480758 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.480711 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" podStartSLOduration=1.995424684 podStartE2EDuration="4.480697133s" totalImagesPullingTime="2.485272449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:55.479543762 +0000 UTC m=+149.229911340" watchObservedRunningTime="2026-07-06 17:40:55.480697133 +0000 UTC m=+149.231064687" Jul 06 17:40:55.501201 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:55.501149 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" podStartSLOduration=1.8054011060000001 podStartE2EDuration="5.500832338s" totalImagesPullingTime="3.695431232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:55.499801415 +0000 UTC m=+149.250168993" watchObservedRunningTime="2026-07-06 17:40:55.500832338 +0000 UTC m=+149.251199916" Jul 06 17:40:56.472879 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:56.472788 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerID="af293ca25e1c5b53d45aaf19e8263659d63c9697d96b1feac6685081af7061d4" exitCode=0 Jul 06 17:40:56.473338 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:56.472876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"af293ca25e1c5b53d45aaf19e8263659d63c9697d96b1feac6685081af7061d4"} Jul 06 17:40:56.475768 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:56.475739 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" event={"ID":"18db26d8-6e39-4e47-a2d2-6c196853c659","Type":"ContainerStarted","Data":"49eef8603581acdcc8612740f700b05592f14db53f0ea7d21518cfa64c648bb7"} Jul 06 17:40:56.475887 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:56.475793 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" event={"ID":"18db26d8-6e39-4e47-a2d2-6c196853c659","Type":"ContainerStarted","Data":"7417f30406b44ba6aa9d59fbf6a2c1692f891be3cf6a243443024eb96f49d6f0"} Jul 06 17:40:56.521671 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:56.521616 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5d7df4f5cc-xswfd" podStartSLOduration=1.294444079 podStartE2EDuration="4.521598613s" totalImagesPullingTime="3.227154534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:56.519532155 +0000 UTC m=+150.269899732" watchObservedRunningTime="2026-07-06 17:40:56.521598613 +0000 UTC m=+150.271966192" Jul 06 17:40:59.859128 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:40:59.859081 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65dc7bfff9-glg9j"] Jul 06 17:40:59.859733 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:40:59.859524 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" podUID="2884e0ee-2837-48cf-abfe-5e98ff81e79a" Jul 06 17:41:00.502149 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.502122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:41:00.502312 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.502269 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerStarted","Data":"b26a9e6030ace1db8cfb46d21e1e477382371a5e3a5a1e8b5e72bf1fc0bc8e64"} Jul 06 17:41:00.518890 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.518849 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:41:00.661403 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.660512 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2884e0ee-2837-48cf-abfe-5e98ff81e79a-ca-trust-extracted\") pod \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " Jul 06 17:41:00.661403 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.660567 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-installation-pull-secrets\") pod \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " Jul 06 17:41:00.661403 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.660624 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-trusted-ca\") pod \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " Jul 06 17:41:00.661403 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.660650 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-649g6\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-kube-api-access-649g6\") pod \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " Jul 06 17:41:00.661403 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.660704 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-bound-sa-token\") pod \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " Jul 06 17:41:00.661403 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.660739 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-image-registry-private-configuration\") pod \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " Jul 06 17:41:00.661403 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.660831 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-certificates\") pod \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\" (UID: \"2884e0ee-2837-48cf-abfe-5e98ff81e79a\") " Jul 06 17:41:00.661905 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.661536 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2884e0ee-2837-48cf-abfe-5e98ff81e79a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2884e0ee-2837-48cf-abfe-5e98ff81e79a" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:41:00.661905 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.661602 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2884e0ee-2837-48cf-abfe-5e98ff81e79a" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:41:00.662195 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.662154 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2884e0ee-2837-48cf-abfe-5e98ff81e79a" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:41:00.664316 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.664220 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2884e0ee-2837-48cf-abfe-5e98ff81e79a" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:41:00.664496 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.664472 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2884e0ee-2837-48cf-abfe-5e98ff81e79a" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:41:00.664760 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.664718 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-kube-api-access-649g6" (OuterVolumeSpecName: "kube-api-access-649g6") pod "2884e0ee-2837-48cf-abfe-5e98ff81e79a" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a"). InnerVolumeSpecName "kube-api-access-649g6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:41:00.665040 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.665004 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2884e0ee-2837-48cf-abfe-5e98ff81e79a" (UID: "2884e0ee-2837-48cf-abfe-5e98ff81e79a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:41:00.762240 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.762213 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-certificates\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:00.762240 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.762244 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2884e0ee-2837-48cf-abfe-5e98ff81e79a-ca-trust-extracted\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:00.762431 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.762259 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-installation-pull-secrets\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:00.762431 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.762273 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2884e0ee-2837-48cf-abfe-5e98ff81e79a-trusted-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:00.762431 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.762303 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-649g6\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-kube-api-access-649g6\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:00.762431 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.762320 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-bound-sa-token\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:00.762431 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:00.762329 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2884e0ee-2837-48cf-abfe-5e98ff81e79a-image-registry-private-configuration\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:01.482597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.482569 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-76f6c55d98-h2czn" Jul 06 17:41:01.513256 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.513219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerStarted","Data":"76fe699f73167f1676e8e9616626f657506068506bc0dcc0749fa75926937e41"} Jul 06 17:41:01.513545 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.513262 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65dc7bfff9-glg9j" Jul 06 17:41:01.513545 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.513319 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerStarted","Data":"8751be25f08f40ee572dd4da64324945457f51b7885dc43450c05f0546c2294c"} Jul 06 17:41:01.513545 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.513363 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerStarted","Data":"92629af9fdd8c3ceff64e7bf785cbfd3b8626eca95ee48825d08a416b96a3b53"} Jul 06 17:41:01.513545 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.513403 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerStarted","Data":"3e40f9cc2b8420b3449bad6497291cb3eb4d6d32b1efe836aad112b4da34e3d6"} Jul 06 17:41:01.513545 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.513442 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerStarted","Data":"efc7b16b81ad4d7e60a99c12c5d59bb59e16c46ce62de734bc2a4764de3ecc92"} Jul 06 17:41:01.550724 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.550663 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.553738301 podStartE2EDuration="8.550641577s" totalImagesPullingTime="4.966863995s" totalInitContainerRuntime="30.039281ms" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:01.548764799 +0000 UTC m=+155.299132399" watchObservedRunningTime="2026-07-06 17:41:01.550641577 +0000 UTC m=+155.301009153" Jul 06 17:41:01.585181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.580612 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65dc7bfff9-glg9j"] Jul 06 17:41:01.585181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.582851 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65dc7bfff9-glg9j"] Jul 06 17:41:01.673202 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:01.672879 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2884e0ee-2837-48cf-abfe-5e98ff81e79a-registry-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:41:02.768161 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:02.768129 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2884e0ee-2837-48cf-abfe-5e98ff81e79a" path="/var/lib/kubelet/pods/2884e0ee-2837-48cf-abfe-5e98ff81e79a/volumes" Jul 06 17:41:03.655857 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:03.655824 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:41:05.322357 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:41:05.322312 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lkl8n" podUID="cc27b535-341b-46dd-9be8-7ce1b75e6e7f" Jul 06 17:41:05.531702 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:05.531668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lkl8n" Jul 06 17:41:10.262068 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:10.262027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:41:10.264719 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:10.264697 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc27b535-341b-46dd-9be8-7ce1b75e6e7f-metrics-tls\") pod \"dns-default-lkl8n\" (UID: \"cc27b535-341b-46dd-9be8-7ce1b75e6e7f\") " pod="openshift-dns/dns-default-lkl8n" Jul 06 17:41:10.335861 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:10.335832 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-75j6z\"" Jul 06 17:41:10.343397 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:10.343360 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lkl8n" Jul 06 17:41:11.810191 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:11.810139 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:41:11.810721 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:11.810215 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:41:12.152194 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:12.152163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lkl8n"] Jul 06 17:41:12.154492 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:41:12.154462 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc27b535_341b_46dd_9be8_7ce1b75e6e7f.slice/crio-cc8d8beeb4b7c9606a1e9880edb57c658654aa20dc01541664531a25c0ea02c6 WatchSource:0}: Error finding container cc8d8beeb4b7c9606a1e9880edb57c658654aa20dc01541664531a25c0ea02c6: Status 404 returned error can't find the container with id cc8d8beeb4b7c9606a1e9880edb57c658654aa20dc01541664531a25c0ea02c6 Jul 06 17:41:12.563917 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:12.563881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lkl8n" event={"ID":"cc27b535-341b-46dd-9be8-7ce1b75e6e7f","Type":"ContainerStarted","Data":"cc8d8beeb4b7c9606a1e9880edb57c658654aa20dc01541664531a25c0ea02c6"} Jul 06 17:41:12.565449 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:12.565424 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-dm7mj" event={"ID":"ba2ffe7e-2e50-42b5-8dca-7f4f2b119381","Type":"ContainerStarted","Data":"ea226f644d77fe7bc4a16b6159c655ec0479952ccd664fef1555c3b5b5d54e12"} Jul 06 17:41:12.565644 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:12.565629 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-dm7mj" Jul 06 17:41:12.566828 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:12.566796 2580 patch_prober.go:28] interesting pod/downloads-685bfb5f64-dm7mj container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.19:8080/\": dial tcp 10.132.0.19:8080: connect: connection refused" start-of-body= Jul 06 17:41:12.566911 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:12.566841 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-dm7mj" podUID="ba2ffe7e-2e50-42b5-8dca-7f4f2b119381" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.19:8080/\": dial tcp 10.132.0.19:8080: connect: connection refused" Jul 06 17:41:12.586178 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:12.586130 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-dm7mj" podStartSLOduration=1.217282287 podStartE2EDuration="21.586116225s" totalImagesPullingTime="20.368833938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:12.585848388 +0000 UTC m=+166.336215965" watchObservedRunningTime="2026-07-06 17:41:12.586116225 +0000 UTC m=+166.336483800" Jul 06 17:41:13.585474 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:13.585434 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-dm7mj" Jul 06 17:41:15.582285 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:15.582223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lkl8n" event={"ID":"cc27b535-341b-46dd-9be8-7ce1b75e6e7f","Type":"ContainerStarted","Data":"75f6a8252612d44558270bd1568e0192f1464d21c867123675b1299a2c1d37d4"} Jul 06 17:41:15.582789 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:15.582411 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lkl8n" event={"ID":"cc27b535-341b-46dd-9be8-7ce1b75e6e7f","Type":"ContainerStarted","Data":"d1b86a2d4b3c708683e389ce8004ba933d2acdc182f2ab554a615a8a9cef96cf"} Jul 06 17:41:15.582789 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:15.582446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lkl8n" Jul 06 17:41:15.602113 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:15.602057 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lkl8n" podStartSLOduration=130.898489167 podStartE2EDuration="2m13.602039976s" totalImagesPullingTime="2.703550809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:39:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:15.600315512 +0000 UTC m=+169.350683084" watchObservedRunningTime="2026-07-06 17:41:15.602039976 +0000 UTC m=+169.352407552" Jul 06 17:41:25.590119 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:25.590032 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lkl8n" Jul 06 17:41:31.815468 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:31.815434 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:41:31.819187 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:31.819166 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6d9668478f-bs6fw" Jul 06 17:41:53.655994 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:53.655958 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:41:53.709246 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:53.709214 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:41:53.763624 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:41:53.763593 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:11.616107 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.616070 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:42:11.617120 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.617052 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="prometheus" containerID="cri-o://b26a9e6030ace1db8cfb46d21e1e477382371a5e3a5a1e8b5e72bf1fc0bc8e64" gracePeriod=600 Jul 06 17:42:11.617338 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.617262 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy" containerID="cri-o://8751be25f08f40ee572dd4da64324945457f51b7885dc43450c05f0546c2294c" gracePeriod=600 Jul 06 17:42:11.617432 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.617329 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-web" containerID="cri-o://92629af9fdd8c3ceff64e7bf785cbfd3b8626eca95ee48825d08a416b96a3b53" gracePeriod=600 Jul 06 17:42:11.617487 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.617437 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="thanos-sidecar" containerID="cri-o://3e40f9cc2b8420b3449bad6497291cb3eb4d6d32b1efe836aad112b4da34e3d6" gracePeriod=600 Jul 06 17:42:11.617487 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.617462 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="config-reloader" containerID="cri-o://efc7b16b81ad4d7e60a99c12c5d59bb59e16c46ce62de734bc2a4764de3ecc92" gracePeriod=600 Jul 06 17:42:11.617583 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.617442 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://76fe699f73167f1676e8e9616626f657506068506bc0dcc0749fa75926937e41" gracePeriod=600 Jul 06 17:42:11.825148 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825113 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerID="76fe699f73167f1676e8e9616626f657506068506bc0dcc0749fa75926937e41" exitCode=0 Jul 06 17:42:11.825148 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825155 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerID="8751be25f08f40ee572dd4da64324945457f51b7885dc43450c05f0546c2294c" exitCode=0 Jul 06 17:42:11.825409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"76fe699f73167f1676e8e9616626f657506068506bc0dcc0749fa75926937e41"} Jul 06 17:42:11.825409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825165 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerID="92629af9fdd8c3ceff64e7bf785cbfd3b8626eca95ee48825d08a416b96a3b53" exitCode=0 Jul 06 17:42:11.825409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825261 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerID="3e40f9cc2b8420b3449bad6497291cb3eb4d6d32b1efe836aad112b4da34e3d6" exitCode=0 Jul 06 17:42:11.825409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825273 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerID="efc7b16b81ad4d7e60a99c12c5d59bb59e16c46ce62de734bc2a4764de3ecc92" exitCode=0 Jul 06 17:42:11.825409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825282 2580 generic.go:356] "Generic (PLEG): container finished" podID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerID="b26a9e6030ace1db8cfb46d21e1e477382371a5e3a5a1e8b5e72bf1fc0bc8e64" exitCode=0 Jul 06 17:42:11.825409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825244 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"8751be25f08f40ee572dd4da64324945457f51b7885dc43450c05f0546c2294c"} Jul 06 17:42:11.825595 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825415 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"92629af9fdd8c3ceff64e7bf785cbfd3b8626eca95ee48825d08a416b96a3b53"} Jul 06 17:42:11.825595 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"3e40f9cc2b8420b3449bad6497291cb3eb4d6d32b1efe836aad112b4da34e3d6"} Jul 06 17:42:11.825595 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825495 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"efc7b16b81ad4d7e60a99c12c5d59bb59e16c46ce62de734bc2a4764de3ecc92"} Jul 06 17:42:11.825595 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.825531 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"b26a9e6030ace1db8cfb46d21e1e477382371a5e3a5a1e8b5e72bf1fc0bc8e64"} Jul 06 17:42:11.868055 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.867984 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:11.909411 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909376 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2jl6\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-kube-api-access-p2jl6\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909434 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-0\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909467 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-kube-rbac-proxy\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909493 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909523 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909565 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config-out\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909587 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-db\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909618 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-thanos-prometheus-http-client-file\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909636 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-grpc-tls\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909660 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-tls\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909678 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-2\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909708 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909728 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-metrics-client-ca\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909746 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-kubelet-serving-ca-bundle\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909766 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-1\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909821 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-tls-assets\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909838 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-serving-certs-ca-bundle\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909861 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-trusted-ca-bundle\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909876 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-web-config\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.909976 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.909900 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-metrics-client-certs\") pod \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\" (UID: \"ef629912-a692-4e12-a7ef-7255d1ed6c2a\") " Jul 06 17:42:11.910871 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.910806 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:42:11.911457 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.911423 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:42:11.912572 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.912196 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:42:11.913095 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.912815 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:42:11.913380 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.913353 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:42:11.914194 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.914165 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:42:11.914284 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.914249 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.914284 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.914259 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:42:11.915710 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.915683 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:42:11.917448 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.917412 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.917743 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.917661 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.917743 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.917680 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config" (OuterVolumeSpecName: "config") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.917743 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.917719 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.917743 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.917740 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config-out" (OuterVolumeSpecName: "config-out") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:42:11.920367 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.919116 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:42:11.920367 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.920217 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.920367 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.920258 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.920367 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.920277 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-kube-api-access-p2jl6" (OuterVolumeSpecName: "kube-api-access-p2jl6") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "kube-api-access-p2jl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:42:11.920605 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.920437 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:11.932433 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:11.932403 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-web-config" (OuterVolumeSpecName: "web-config") pod "ef629912-a692-4e12-a7ef-7255d1ed6c2a" (UID: "ef629912-a692-4e12-a7ef-7255d1ed6c2a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:42:12.011323 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011252 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-tls-assets\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011323 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011285 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011323 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011325 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011339 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-web-config\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011351 2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-metrics-client-certs\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011363 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2jl6\" (UniqueName: \"kubernetes.io/projected/ef629912-a692-4e12-a7ef-7255d1ed6c2a-kube-api-access-p2jl6\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011374 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011386 2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-kube-rbac-proxy\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011399 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011412 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011424 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config-out\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011438 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-db\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011450 2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011462 2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-grpc-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011474 2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011487 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011499 2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef629912-a692-4e12-a7ef-7255d1ed6c2a-config\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011510 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-metrics-client-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011523 2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.011603 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.011535 2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef629912-a692-4e12-a7ef-7255d1ed6c2a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:42:12.833042 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.833005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef629912-a692-4e12-a7ef-7255d1ed6c2a","Type":"ContainerDied","Data":"68d7838aeadd66ec86ed6807315f777b63046802ee51d79c0cdd6b2d4a82b79e"} Jul 06 17:42:12.833465 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.833092 2580 scope.go:117] "RemoveContainer" containerID="76fe699f73167f1676e8e9616626f657506068506bc0dcc0749fa75926937e41" Jul 06 17:42:12.833465 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.833019 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:12.842055 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.842036 2580 scope.go:117] "RemoveContainer" containerID="8751be25f08f40ee572dd4da64324945457f51b7885dc43450c05f0546c2294c" Jul 06 17:42:12.849266 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.849242 2580 scope.go:117] "RemoveContainer" containerID="92629af9fdd8c3ceff64e7bf785cbfd3b8626eca95ee48825d08a416b96a3b53" Jul 06 17:42:12.856482 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.856460 2580 scope.go:117] "RemoveContainer" containerID="3e40f9cc2b8420b3449bad6497291cb3eb4d6d32b1efe836aad112b4da34e3d6" Jul 06 17:42:12.857788 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.857769 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:42:12.860625 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.860606 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:42:12.863988 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.863970 2580 scope.go:117] "RemoveContainer" containerID="efc7b16b81ad4d7e60a99c12c5d59bb59e16c46ce62de734bc2a4764de3ecc92" Jul 06 17:42:12.870781 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.870761 2580 scope.go:117] "RemoveContainer" containerID="b26a9e6030ace1db8cfb46d21e1e477382371a5e3a5a1e8b5e72bf1fc0bc8e64" Jul 06 17:42:12.880997 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.880978 2580 scope.go:117] "RemoveContainer" containerID="af293ca25e1c5b53d45aaf19e8263659d63c9697d96b1feac6685081af7061d4" Jul 06 17:42:12.926528 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926498 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:42:12.926801 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926789 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-web" Jul 06 17:42:12.926845 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926803 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-web" Jul 06 17:42:12.926845 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926813 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-thanos" Jul 06 17:42:12.926845 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926819 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-thanos" Jul 06 17:42:12.926845 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926827 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="prometheus" Jul 06 17:42:12.926845 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926833 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="prometheus" Jul 06 17:42:12.926845 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926843 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="thanos-sidecar" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926849 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="thanos-sidecar" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926858 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="init-config-reloader" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926863 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="init-config-reloader" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926869 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926875 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926882 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="config-reloader" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926888 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="config-reloader" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926949 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-web" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926957 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="prometheus" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926965 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="config-reloader" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926970 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="thanos-sidecar" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926976 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy-thanos" Jul 06 17:42:12.927013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.926982 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" containerName="kube-rbac-proxy" Jul 06 17:42:12.930454 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.930438 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:12.933158 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933129 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 17:42:12.933632 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933612 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 17:42:12.933715 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933641 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 17:42:12.933881 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933828 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 17:42:12.934014 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933900 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 17:42:12.934014 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933942 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 17:42:12.934014 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933900 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 17:42:12.934014 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933952 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 17:42:12.934014 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.933980 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2tx77\"" Jul 06 17:42:12.934329 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.934127 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c93125qn5pdqi\"" Jul 06 17:42:12.934329 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.934203 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 17:42:12.934329 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.934235 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 17:42:12.934329 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.934260 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 17:42:12.934828 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.934809 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 17:42:12.936826 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.936782 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 17:42:12.940086 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.940069 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 17:42:12.947445 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:12.947422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:42:13.018690 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-config\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018690 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707c78bf-5b26-403c-b11f-5b68285e0ee8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018880 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018899 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-web-config\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.018933 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707c78bf-5b26-403c-b11f-5b68285e0ee8-config-out\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.018971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019076 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.019219 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.019190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qjqv\" (UniqueName: \"kubernetes.io/projected/707c78bf-5b26-403c-b11f-5b68285e0ee8-kube-api-access-2qjqv\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.119821 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.119783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-config\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.119821 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.119825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.119843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.119863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.119991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707c78bf-5b26-403c-b11f-5b68285e0ee8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120070 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-web-config\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707c78bf-5b26-403c-b11f-5b68285e0ee8-config-out\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120370 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120434 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120853 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120853 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120853 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qjqv\" (UniqueName: \"kubernetes.io/projected/707c78bf-5b26-403c-b11f-5b68285e0ee8-kube-api-access-2qjqv\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.120853 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.120843 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.121480 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.121453 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.121581 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.121516 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.121812 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.121783 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.122331 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.122307 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.123555 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.122917 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/707c78bf-5b26-403c-b11f-5b68285e0ee8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.123555 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.123127 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-config\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124190 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.123851 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124190 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.123910 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124190 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.123932 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124190 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.124062 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124190 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.124096 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124507 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.124395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/707c78bf-5b26-403c-b11f-5b68285e0ee8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124582 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.124559 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-web-config\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.124844 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.124824 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.125899 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.125872 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/707c78bf-5b26-403c-b11f-5b68285e0ee8-config-out\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.126146 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.126128 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.126613 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.126591 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.126911 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.126889 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/707c78bf-5b26-403c-b11f-5b68285e0ee8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.128445 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.128424 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qjqv\" (UniqueName: \"kubernetes.io/projected/707c78bf-5b26-403c-b11f-5b68285e0ee8-kube-api-access-2qjqv\") pod \"prometheus-k8s-0\" (UID: \"707c78bf-5b26-403c-b11f-5b68285e0ee8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.239996 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.239961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:42:13.376419 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.376225 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:42:13.378588 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:42:13.378556 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707c78bf_5b26_403c_b11f_5b68285e0ee8.slice/crio-7b0a737ba6f55465d9fba1c0119bcf5b0413f12f65dc261e123138b4472b7802 WatchSource:0}: Error finding container 7b0a737ba6f55465d9fba1c0119bcf5b0413f12f65dc261e123138b4472b7802: Status 404 returned error can't find the container with id 7b0a737ba6f55465d9fba1c0119bcf5b0413f12f65dc261e123138b4472b7802 Jul 06 17:42:13.839246 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.839210 2580 generic.go:356] "Generic (PLEG): container finished" podID="707c78bf-5b26-403c-b11f-5b68285e0ee8" containerID="73964a65087cfb59f9293a6668c050d2a62eae9285a76d12de7a40aa25a59a2b" exitCode=0 Jul 06 17:42:13.839763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.839286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerDied","Data":"73964a65087cfb59f9293a6668c050d2a62eae9285a76d12de7a40aa25a59a2b"} Jul 06 17:42:13.839763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:13.839438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerStarted","Data":"7b0a737ba6f55465d9fba1c0119bcf5b0413f12f65dc261e123138b4472b7802"} Jul 06 17:42:14.768599 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.768558 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef629912-a692-4e12-a7ef-7255d1ed6c2a" path="/var/lib/kubelet/pods/ef629912-a692-4e12-a7ef-7255d1ed6c2a/volumes" Jul 06 17:42:14.846570 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.846539 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerStarted","Data":"1b7dc323efc5ca85a4c812349617542e6ad8261ab5bf3e9a1d709e4d04ee0b06"} Jul 06 17:42:14.846953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.846601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerStarted","Data":"4e4474128cf3ddf3fa669a29f619c4082d7da64f81d6a055ff657cd0eaf50d24"} Jul 06 17:42:14.846953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.846628 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerStarted","Data":"19a238b50067669e0e8d5943ef5a600ed69c06be43ecc6a2560d9892039cd7d4"} Jul 06 17:42:14.846953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.846657 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerStarted","Data":"d4f7ad4e32ca1fd1c0d3cef61147ed2d917941706269d29345fbc44580dbd3cd"} Jul 06 17:42:14.846953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.846681 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerStarted","Data":"701eae234acddec78f4dca2f97b49b64f98806bf13b9d472104888c66e1dc17a"} Jul 06 17:42:14.846953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.846708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"707c78bf-5b26-403c-b11f-5b68285e0ee8","Type":"ContainerStarted","Data":"fead5960d9d0d0db4f9f9242f87f66d988bfaa639948a3854efff53c6700e133"} Jul 06 17:42:14.879479 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:14.879430 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.8628865340000003 podStartE2EDuration="2.879408894s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.52236ms" isStatefulPod=false podCreationTimestamp="2026-07-06 17:42:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:42:14.87684549 +0000 UTC m=+228.627213087" watchObservedRunningTime="2026-07-06 17:42:14.879408894 +0000 UTC m=+228.629776470" Jul 06 17:42:18.240458 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:42:18.240422 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:43:13.241015 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:43:13.240973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:43:13.256826 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:43:13.256797 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:43:14.099615 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:43:14.099585 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:43:26.628641 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:43:26.628605 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:43:26.636082 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:43:26.636057 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 17:43:26.636453 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:43:26.636437 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:44:35.729145 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.729063 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll"] Jul 06 17:44:35.731322 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.731286 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:35.734178 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.734158 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:44:35.735388 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.735369 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 17:44:35.735496 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.735393 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-x6kpd\"" Jul 06 17:44:35.740224 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.740201 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll"] Jul 06 17:44:35.838472 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.838441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llssn\" (UniqueName: \"kubernetes.io/projected/d9b6db0c-d1be-492b-b2ff-8581db2597e2-kube-api-access-llssn\") pod \"openshift-lws-operator-fd8ccff4c-hllll\" (UID: \"d9b6db0c-d1be-492b-b2ff-8581db2597e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:35.838641 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.838507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9b6db0c-d1be-492b-b2ff-8581db2597e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hllll\" (UID: \"d9b6db0c-d1be-492b-b2ff-8581db2597e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:35.939711 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.939682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9b6db0c-d1be-492b-b2ff-8581db2597e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hllll\" (UID: \"d9b6db0c-d1be-492b-b2ff-8581db2597e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:35.940063 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.940044 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9b6db0c-d1be-492b-b2ff-8581db2597e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hllll\" (UID: \"d9b6db0c-d1be-492b-b2ff-8581db2597e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:35.940154 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.940063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llssn\" (UniqueName: \"kubernetes.io/projected/d9b6db0c-d1be-492b-b2ff-8581db2597e2-kube-api-access-llssn\") pod \"openshift-lws-operator-fd8ccff4c-hllll\" (UID: \"d9b6db0c-d1be-492b-b2ff-8581db2597e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:35.948495 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:35.948475 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llssn\" (UniqueName: \"kubernetes.io/projected/d9b6db0c-d1be-492b-b2ff-8581db2597e2-kube-api-access-llssn\") pod \"openshift-lws-operator-fd8ccff4c-hllll\" (UID: \"d9b6db0c-d1be-492b-b2ff-8581db2597e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:36.058134 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:36.058048 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" Jul 06 17:44:36.181778 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:36.181756 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll"] Jul 06 17:44:36.183957 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:44:36.183928 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9b6db0c_d1be_492b_b2ff_8581db2597e2.slice/crio-f54943274bb11b1372cb5c9bc27c359f0e509e641dd3cde18a61300b39e3e3d2 WatchSource:0}: Error finding container f54943274bb11b1372cb5c9bc27c359f0e509e641dd3cde18a61300b39e3e3d2: Status 404 returned error can't find the container with id f54943274bb11b1372cb5c9bc27c359f0e509e641dd3cde18a61300b39e3e3d2 Jul 06 17:44:36.185413 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:36.185396 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 17:44:36.412112 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:36.412015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" event={"ID":"d9b6db0c-d1be-492b-b2ff-8581db2597e2","Type":"ContainerStarted","Data":"f54943274bb11b1372cb5c9bc27c359f0e509e641dd3cde18a61300b39e3e3d2"} Jul 06 17:44:43.443372 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:43.443328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" event={"ID":"d9b6db0c-d1be-492b-b2ff-8581db2597e2","Type":"ContainerStarted","Data":"d88d724b866b2d4afefa291adde19e5dc74c348adac3d66cf8b694ea712dd096"} Jul 06 17:44:43.461015 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:43.460919 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hllll" podStartSLOduration=1.437787213 podStartE2EDuration="8.460904619s" totalImagesPullingTime="7.023117406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:44:43.459458163 +0000 UTC m=+377.209825756" watchObservedRunningTime="2026-07-06 17:44:43.460904619 +0000 UTC m=+377.211272195" Jul 06 17:44:58.440634 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.440597 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm"] Jul 06 17:44:58.443939 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.443917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.446860 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.446838 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7qxkd\"" Jul 06 17:44:58.448398 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.448380 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 17:44:58.448490 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.448399 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 17:44:58.448490 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.448380 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 17:44:58.456088 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.456060 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm"] Jul 06 17:44:58.536229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.536197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/622b26a0-5ac6-43bf-afb5-b4bec79c5981-manager-config\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.536229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.536235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtfbg\" (UniqueName: \"kubernetes.io/projected/622b26a0-5ac6-43bf-afb5-b4bec79c5981-kube-api-access-wtfbg\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.536481 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.536279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/622b26a0-5ac6-43bf-afb5-b4bec79c5981-metrics-cert\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.536481 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.536369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/622b26a0-5ac6-43bf-afb5-b4bec79c5981-cert\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.637716 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.637677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtfbg\" (UniqueName: \"kubernetes.io/projected/622b26a0-5ac6-43bf-afb5-b4bec79c5981-kube-api-access-wtfbg\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.637916 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.637741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/622b26a0-5ac6-43bf-afb5-b4bec79c5981-metrics-cert\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.637916 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.637849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/622b26a0-5ac6-43bf-afb5-b4bec79c5981-cert\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.638029 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.637933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/622b26a0-5ac6-43bf-afb5-b4bec79c5981-manager-config\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.638536 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.638515 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/622b26a0-5ac6-43bf-afb5-b4bec79c5981-manager-config\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.640318 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.640267 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/622b26a0-5ac6-43bf-afb5-b4bec79c5981-cert\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.640402 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.640364 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/622b26a0-5ac6-43bf-afb5-b4bec79c5981-metrics-cert\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.653665 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.653642 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtfbg\" (UniqueName: \"kubernetes.io/projected/622b26a0-5ac6-43bf-afb5-b4bec79c5981-kube-api-access-wtfbg\") pod \"lws-controller-manager-68c469dbcb-n4srm\" (UID: \"622b26a0-5ac6-43bf-afb5-b4bec79c5981\") " pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.753835 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.753789 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:44:58.892012 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:58.891981 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm"] Jul 06 17:44:58.893374 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:44:58.893348 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622b26a0_5ac6_43bf_afb5_b4bec79c5981.slice/crio-2c5962fa44e1b85a73f88a92ea312862e37db64d263926de0c6d2e98c61e7061 WatchSource:0}: Error finding container 2c5962fa44e1b85a73f88a92ea312862e37db64d263926de0c6d2e98c61e7061: Status 404 returned error can't find the container with id 2c5962fa44e1b85a73f88a92ea312862e37db64d263926de0c6d2e98c61e7061 Jul 06 17:44:59.284525 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.284487 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2"] Jul 06 17:44:59.289094 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.289070 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.293092 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.293072 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 17:44:59.293092 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.293087 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 17:44:59.293321 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.293309 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bk52m\"" Jul 06 17:44:59.293589 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.293576 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 17:44:59.294348 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.294334 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 17:44:59.311770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.311738 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2"] Jul 06 17:44:59.345554 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.345525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba8ba816-0274-411b-bf7f-6269530ff2cb-manifests\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.345554 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.345556 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba8ba816-0274-411b-bf7f-6269530ff2cb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.345763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.345597 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba8ba816-0274-411b-bf7f-6269530ff2cb-tmp\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.345763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.345633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7brb\" (UniqueName: \"kubernetes.io/projected/ba8ba816-0274-411b-bf7f-6269530ff2cb-kube-api-access-m7brb\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.345763 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.345715 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba8ba816-0274-411b-bf7f-6269530ff2cb-webhook-cert\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.446724 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.446678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba8ba816-0274-411b-bf7f-6269530ff2cb-webhook-cert\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.447181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.446733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba8ba816-0274-411b-bf7f-6269530ff2cb-manifests\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.447181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.446755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba8ba816-0274-411b-bf7f-6269530ff2cb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.447181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.446791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba8ba816-0274-411b-bf7f-6269530ff2cb-tmp\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.447181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.446847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7brb\" (UniqueName: \"kubernetes.io/projected/ba8ba816-0274-411b-bf7f-6269530ff2cb-kube-api-access-m7brb\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.447181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.447102 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba8ba816-0274-411b-bf7f-6269530ff2cb-manifests\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.449072 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.449042 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba8ba816-0274-411b-bf7f-6269530ff2cb-tmp\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.449281 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.449262 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba8ba816-0274-411b-bf7f-6269530ff2cb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.449427 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.449408 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba8ba816-0274-411b-bf7f-6269530ff2cb-webhook-cert\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.462573 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.462554 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7brb\" (UniqueName: \"kubernetes.io/projected/ba8ba816-0274-411b-bf7f-6269530ff2cb-kube-api-access-m7brb\") pod \"opendatahub-operator-controller-manager-597bcb8998-btmg2\" (UID: \"ba8ba816-0274-411b-bf7f-6269530ff2cb\") " pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.508066 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.508026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" event={"ID":"622b26a0-5ac6-43bf-afb5-b4bec79c5981","Type":"ContainerStarted","Data":"2c5962fa44e1b85a73f88a92ea312862e37db64d263926de0c6d2e98c61e7061"} Jul 06 17:44:59.600577 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.600489 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:44:59.734494 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:44:59.734468 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2"] Jul 06 17:44:59.736851 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:44:59.736817 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8ba816_0274_411b_bf7f_6269530ff2cb.slice/crio-75271b2bebed2947f2cccad958a15e640ec90748f3448cd12678d17e5f2fb68e WatchSource:0}: Error finding container 75271b2bebed2947f2cccad958a15e640ec90748f3448cd12678d17e5f2fb68e: Status 404 returned error can't find the container with id 75271b2bebed2947f2cccad958a15e640ec90748f3448cd12678d17e5f2fb68e Jul 06 17:45:00.518049 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:00.518010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" event={"ID":"ba8ba816-0274-411b-bf7f-6269530ff2cb","Type":"ContainerStarted","Data":"75271b2bebed2947f2cccad958a15e640ec90748f3448cd12678d17e5f2fb68e"} Jul 06 17:45:01.526868 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:01.526833 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" event={"ID":"622b26a0-5ac6-43bf-afb5-b4bec79c5981","Type":"ContainerStarted","Data":"825c55b46a008ba6aefb4663e20e34c6276f83cd86261fbf07a55d080255ab41"} Jul 06 17:45:01.527366 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:01.526999 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:45:01.549774 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:01.549708 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" podStartSLOduration=1.48457887 podStartE2EDuration="3.549686269s" totalImagesPullingTime="2.065107399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:01.547618468 +0000 UTC m=+395.297986047" watchObservedRunningTime="2026-07-06 17:45:01.549686269 +0000 UTC m=+395.300053848" Jul 06 17:45:03.537240 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:03.537199 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" event={"ID":"ba8ba816-0274-411b-bf7f-6269530ff2cb","Type":"ContainerStarted","Data":"d12d3e5a20f6349d0ba96d8ea4ebe76ed619b73349f46291235c38078e5b177d"} Jul 06 17:45:04.542727 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:04.542694 2580 generic.go:356] "Generic (PLEG): container finished" podID="ba8ba816-0274-411b-bf7f-6269530ff2cb" containerID="d12d3e5a20f6349d0ba96d8ea4ebe76ed619b73349f46291235c38078e5b177d" exitCode=0 Jul 06 17:45:04.543136 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:04.542781 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" event={"ID":"ba8ba816-0274-411b-bf7f-6269530ff2cb","Type":"ContainerDied","Data":"d12d3e5a20f6349d0ba96d8ea4ebe76ed619b73349f46291235c38078e5b177d"} Jul 06 17:45:05.550675 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:05.550632 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" event={"ID":"ba8ba816-0274-411b-bf7f-6269530ff2cb","Type":"ContainerStarted","Data":"ca6c60ad7975a4ff756ccda0a1ec6c66e6b0d1f27563a8364e90259951a62ebb"} Jul 06 17:45:05.551137 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:05.550729 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:45:05.573159 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:05.573101 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" podStartSLOduration=2.096255041 podStartE2EDuration="6.573086232s" totalImagesPullingTime="3.448462364s" totalInitContainerRuntime="1.028368827s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:05.570429327 +0000 UTC m=+399.320796906" watchObservedRunningTime="2026-07-06 17:45:05.573086232 +0000 UTC m=+399.323453808" Jul 06 17:45:12.533638 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:12.533607 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68c469dbcb-n4srm" Jul 06 17:45:16.557758 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:16.557724 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 17:45:18.933478 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.933440 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq"] Jul 06 17:45:18.937450 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.937424 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:18.940794 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.940770 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 17:45:18.940915 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.940803 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 17:45:18.942140 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.942120 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-g7nsn\"" Jul 06 17:45:18.942471 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.942446 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 17:45:18.942471 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.942459 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 17:45:18.948441 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:18.948418 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq"] Jul 06 17:45:19.022918 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.022887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a6aba22-66bb-4428-9717-4f35b33ed960-tmp\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.023106 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.022930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfczd\" (UniqueName: \"kubernetes.io/projected/4a6aba22-66bb-4428-9717-4f35b33ed960-kube-api-access-lfczd\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.023106 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.022981 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a6aba22-66bb-4428-9717-4f35b33ed960-tls-certs\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.124015 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.123985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a6aba22-66bb-4428-9717-4f35b33ed960-tmp\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.124187 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.124035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfczd\" (UniqueName: \"kubernetes.io/projected/4a6aba22-66bb-4428-9717-4f35b33ed960-kube-api-access-lfczd\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.124187 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.124152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a6aba22-66bb-4428-9717-4f35b33ed960-tls-certs\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.126359 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.126327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a6aba22-66bb-4428-9717-4f35b33ed960-tmp\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.126722 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.126694 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a6aba22-66bb-4428-9717-4f35b33ed960-tls-certs\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.136065 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.136035 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfczd\" (UniqueName: \"kubernetes.io/projected/4a6aba22-66bb-4428-9717-4f35b33ed960-kube-api-access-lfczd\") pod \"kube-auth-proxy-6cf85c8449-qhxjq\" (UID: \"4a6aba22-66bb-4428-9717-4f35b33ed960\") " pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.249251 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.249210 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" Jul 06 17:45:19.400395 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.400353 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq"] Jul 06 17:45:19.404189 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:45:19.404164 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a6aba22_66bb_4428_9717_4f35b33ed960.slice/crio-eeac3989646f18a6398137228ca406b5e665ee0e22592a5d97abf546099da6a3 WatchSource:0}: Error finding container eeac3989646f18a6398137228ca406b5e665ee0e22592a5d97abf546099da6a3: Status 404 returned error can't find the container with id eeac3989646f18a6398137228ca406b5e665ee0e22592a5d97abf546099da6a3 Jul 06 17:45:19.620513 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:19.620431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" event={"ID":"4a6aba22-66bb-4428-9717-4f35b33ed960","Type":"ContainerStarted","Data":"eeac3989646f18a6398137228ca406b5e665ee0e22592a5d97abf546099da6a3"} Jul 06 17:45:22.636801 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:22.636767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" event={"ID":"4a6aba22-66bb-4428-9717-4f35b33ed960","Type":"ContainerStarted","Data":"a88ec9cd7f74d45b1baa816e6244e9cce97ba1283dac0a1e9e1a7d65060fee04"} Jul 06 17:45:22.654088 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:45:22.654027 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6cf85c8449-qhxjq" podStartSLOduration=1.494809242 podStartE2EDuration="4.654004964s" totalImagesPullingTime="3.159195722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:22.652728806 +0000 UTC m=+416.403096384" watchObservedRunningTime="2026-07-06 17:45:22.654004964 +0000 UTC m=+416.404372542" Jul 06 17:47:10.245854 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.245818 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn"] Jul 06 17:47:10.249797 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.249776 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.252749 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.252725 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 17:47:10.252749 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.252741 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 17:47:10.252921 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.252760 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 17:47:10.253051 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.253034 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 17:47:10.254304 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.254271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lp5dx\"" Jul 06 17:47:10.256871 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.256850 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn"] Jul 06 17:47:10.327571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.327533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.327752 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.327587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.327752 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.327647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4wfc\" (UniqueName: \"kubernetes.io/projected/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-kube-api-access-r4wfc\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.428361 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.428320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.428546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.428397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.428546 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.428429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4wfc\" (UniqueName: \"kubernetes.io/projected/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-kube-api-access-r4wfc\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.428936 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.428913 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.430756 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.430737 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.437181 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.437150 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4wfc\" (UniqueName: \"kubernetes.io/projected/4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3-kube-api-access-r4wfc\") pod \"kuadrant-console-plugin-66b5774985-njtcn\" (UID: \"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.560485 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.560396 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" Jul 06 17:47:10.693641 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:10.693612 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn"] Jul 06 17:47:10.695913 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:47:10.695880 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d6a3ec5_a544_43eb_ae9a_3fa8fcfecbf3.slice/crio-1b78859a57f3999205d2048b6b39cf909b09f3281a24422b220393c1a8650677 WatchSource:0}: Error finding container 1b78859a57f3999205d2048b6b39cf909b09f3281a24422b220393c1a8650677: Status 404 returned error can't find the container with id 1b78859a57f3999205d2048b6b39cf909b09f3281a24422b220393c1a8650677 Jul 06 17:47:11.104965 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:11.104929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" event={"ID":"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3","Type":"ContainerStarted","Data":"1b78859a57f3999205d2048b6b39cf909b09f3281a24422b220393c1a8650677"} Jul 06 17:47:20.188974 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.188879 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7"] Jul 06 17:47:20.196043 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.196015 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.199191 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.199170 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-82b8x\"" Jul 06 17:47:20.204456 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.204396 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7"] Jul 06 17:47:20.323409 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.323371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4wsd\" (UniqueName: \"kubernetes.io/projected/4272f132-238e-4c17-b4ce-a4f037d02abc-kube-api-access-k4wsd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.323666 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.323442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4272f132-238e-4c17-b4ce-a4f037d02abc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.424462 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.424418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4wsd\" (UniqueName: \"kubernetes.io/projected/4272f132-238e-4c17-b4ce-a4f037d02abc-kube-api-access-k4wsd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.424649 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.424478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4272f132-238e-4c17-b4ce-a4f037d02abc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.424985 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.424962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4272f132-238e-4c17-b4ce-a4f037d02abc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.435249 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.435217 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4wsd\" (UniqueName: \"kubernetes.io/projected/4272f132-238e-4c17-b4ce-a4f037d02abc-kube-api-access-k4wsd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.509915 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.509878 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:20.826634 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.826542 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7"] Jul 06 17:47:20.836799 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:20.836726 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7"] Jul 06 17:47:33.544173 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.544136 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8"] Jul 06 17:47:33.584501 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.584470 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8"] Jul 06 17:47:33.584656 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.584584 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:33.748844 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.748801 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfscf\" (UniqueName: \"kubernetes.io/projected/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-kube-api-access-qfscf\") pod \"kuadrant-operator-controller-manager-86878c769f-gnts8\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:33.749023 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.748855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gnts8\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:33.849955 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.849917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfscf\" (UniqueName: \"kubernetes.io/projected/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-kube-api-access-qfscf\") pod \"kuadrant-operator-controller-manager-86878c769f-gnts8\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:33.850120 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.849960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gnts8\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:33.850350 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.850329 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gnts8\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:33.864063 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.864039 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfscf\" (UniqueName: \"kubernetes.io/projected/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-kube-api-access-qfscf\") pod \"kuadrant-operator-controller-manager-86878c769f-gnts8\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:33.895023 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:33.894995 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:34.030710 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:34.030684 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8"] Jul 06 17:47:34.032524 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:47:34.032495 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb21d8f6_8707_4649_8ee4_a88da16bbb7d.slice/crio-9afc0a9c76a69a69daf5f25bd0a00cb9be4919ac9d62b1ae856a8778489f1f29 WatchSource:0}: Error finding container 9afc0a9c76a69a69daf5f25bd0a00cb9be4919ac9d62b1ae856a8778489f1f29: Status 404 returned error can't find the container with id 9afc0a9c76a69a69daf5f25bd0a00cb9be4919ac9d62b1ae856a8778489f1f29 Jul 06 17:47:34.229831 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:34.229801 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" event={"ID":"bb21d8f6-8707-4649-8ee4-a88da16bbb7d","Type":"ContainerStarted","Data":"9afc0a9c76a69a69daf5f25bd0a00cb9be4919ac9d62b1ae856a8778489f1f29"} Jul 06 17:47:34.231333 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:34.231284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" event={"ID":"4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3","Type":"ContainerStarted","Data":"6d2f38cfed360fd8b525a87e1e3473101d01871ace9d1aa94426c6466dbe2297"} Jul 06 17:47:34.264618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:34.264572 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-njtcn" podStartSLOduration=1.082638221 podStartE2EDuration="24.264557191s" totalImagesPullingTime="23.18191897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:47:34.263538162 +0000 UTC m=+548.013905738" watchObservedRunningTime="2026-07-06 17:47:34.264557191 +0000 UTC m=+548.014924767" Jul 06 17:47:36.353069 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:47:36.353029 2580 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-lgxt7_kuadrant-system_4272f132-238e-4c17-b4ce-a4f037d02abc_0(13cf9b633975ea3232129bb9fbb111974e0fccec7eeb82c8c91b6200d8620fbf): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-lgxt7 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 06 17:47:36.353455 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:47:36.353096 2580 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-lgxt7_kuadrant-system_4272f132-238e-4c17-b4ce-a4f037d02abc_0(13cf9b633975ea3232129bb9fbb111974e0fccec7eeb82c8c91b6200d8620fbf): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-lgxt7 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:37.247544 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.247512 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:37.250108 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:47:37.250072 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" is forbidden: User \"system:node:ip-10-0-138-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-108.ec2.internal' and this object" podUID="4272f132-238e-4c17-b4ce-a4f037d02abc" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:37.266195 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.266169 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:37.268883 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:47:37.268849 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" is forbidden: User \"system:node:ip-10-0-138-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-108.ec2.internal' and this object" podUID="4272f132-238e-4c17-b4ce-a4f037d02abc" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:37.279256 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.279234 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4wsd\" (UniqueName: \"kubernetes.io/projected/4272f132-238e-4c17-b4ce-a4f037d02abc-kube-api-access-k4wsd\") pod \"4272f132-238e-4c17-b4ce-a4f037d02abc\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " Jul 06 17:47:37.279408 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.279273 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4272f132-238e-4c17-b4ce-a4f037d02abc-extensions-socket-volume\") pod \"4272f132-238e-4c17-b4ce-a4f037d02abc\" (UID: \"4272f132-238e-4c17-b4ce-a4f037d02abc\") " Jul 06 17:47:37.279618 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.279592 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4272f132-238e-4c17-b4ce-a4f037d02abc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4272f132-238e-4c17-b4ce-a4f037d02abc" (UID: "4272f132-238e-4c17-b4ce-a4f037d02abc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:47:37.281613 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.281587 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4272f132-238e-4c17-b4ce-a4f037d02abc-kube-api-access-k4wsd" (OuterVolumeSpecName: "kube-api-access-k4wsd") pod "4272f132-238e-4c17-b4ce-a4f037d02abc" (UID: "4272f132-238e-4c17-b4ce-a4f037d02abc"). InnerVolumeSpecName "kube-api-access-k4wsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:47:37.380042 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.380008 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4wsd\" (UniqueName: \"kubernetes.io/projected/4272f132-238e-4c17-b4ce-a4f037d02abc-kube-api-access-k4wsd\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:47:37.380042 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:37.380043 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4272f132-238e-4c17-b4ce-a4f037d02abc-extensions-socket-volume\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:47:38.254135 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:38.254102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:38.257503 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:47:38.257461 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" is forbidden: User \"system:node:ip-10-0-138-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-108.ec2.internal' and this object" podUID="4272f132-238e-4c17-b4ce-a4f037d02abc" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:38.264432 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:47:38.264403 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgxt7\" is forbidden: User \"system:node:ip-10-0-138-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-108.ec2.internal' and this object" podUID="4272f132-238e-4c17-b4ce-a4f037d02abc" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgxt7" Jul 06 17:47:38.769438 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:38.769398 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4272f132-238e-4c17-b4ce-a4f037d02abc" path="/var/lib/kubelet/pods/4272f132-238e-4c17-b4ce-a4f037d02abc/volumes" Jul 06 17:47:40.264853 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:40.264816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" event={"ID":"bb21d8f6-8707-4649-8ee4-a88da16bbb7d","Type":"ContainerStarted","Data":"5e00dc36a5365a9d8c632d9b836dccbfff38cac002b1bab768228991e774c183"} Jul 06 17:47:40.265321 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:40.264960 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:47:40.286824 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:40.286772 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" podStartSLOduration=1.901055374 podStartE2EDuration="7.286757898s" totalImagesPullingTime="5.385702524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:47:40.285898973 +0000 UTC m=+554.036266570" watchObservedRunningTime="2026-07-06 17:47:40.286757898 +0000 UTC m=+554.037125527" Jul 06 17:47:51.272054 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:47:51.272024 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:48:10.102803 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.102765 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-dh4xv"] Jul 06 17:48:10.422929 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.422850 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dh4xv"] Jul 06 17:48:10.423078 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.422947 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dh4xv" Jul 06 17:48:10.427052 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.427016 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-294km\"" Jul 06 17:48:10.578321 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.578274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khngk\" (UniqueName: \"kubernetes.io/projected/16a07cf5-b126-4b65-8026-aefc9a946ebb-kube-api-access-khngk\") pod \"authorino-75d556bd77-dh4xv\" (UID: \"16a07cf5-b126-4b65-8026-aefc9a946ebb\") " pod="kuadrant-system/authorino-75d556bd77-dh4xv" Jul 06 17:48:10.679223 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.679135 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khngk\" (UniqueName: \"kubernetes.io/projected/16a07cf5-b126-4b65-8026-aefc9a946ebb-kube-api-access-khngk\") pod \"authorino-75d556bd77-dh4xv\" (UID: \"16a07cf5-b126-4b65-8026-aefc9a946ebb\") " pod="kuadrant-system/authorino-75d556bd77-dh4xv" Jul 06 17:48:10.687129 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.687105 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khngk\" (UniqueName: \"kubernetes.io/projected/16a07cf5-b126-4b65-8026-aefc9a946ebb-kube-api-access-khngk\") pod \"authorino-75d556bd77-dh4xv\" (UID: \"16a07cf5-b126-4b65-8026-aefc9a946ebb\") " pod="kuadrant-system/authorino-75d556bd77-dh4xv" Jul 06 17:48:10.733088 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.733051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dh4xv" Jul 06 17:48:10.881889 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:10.881869 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dh4xv"] Jul 06 17:48:10.884458 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:48:10.884431 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16a07cf5_b126_4b65_8026_aefc9a946ebb.slice/crio-11545a50f34b71019de16ac8e0842487a82646de37a03806c8feb8d9433f9928 WatchSource:0}: Error finding container 11545a50f34b71019de16ac8e0842487a82646de37a03806c8feb8d9433f9928: Status 404 returned error can't find the container with id 11545a50f34b71019de16ac8e0842487a82646de37a03806c8feb8d9433f9928 Jul 06 17:48:11.404543 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:11.404504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dh4xv" event={"ID":"16a07cf5-b126-4b65-8026-aefc9a946ebb","Type":"ContainerStarted","Data":"11545a50f34b71019de16ac8e0842487a82646de37a03806c8feb8d9433f9928"} Jul 06 17:48:14.423096 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:14.423053 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dh4xv" event={"ID":"16a07cf5-b126-4b65-8026-aefc9a946ebb","Type":"ContainerStarted","Data":"7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1"} Jul 06 17:48:14.437984 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:14.437928 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-dh4xv" podStartSLOduration=1.395022121 podStartE2EDuration="4.437912535s" totalImagesPullingTime="3.042890414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:48:14.437061719 +0000 UTC m=+588.187429294" watchObservedRunningTime="2026-07-06 17:48:14.437912535 +0000 UTC m=+588.188280110" Jul 06 17:48:26.660482 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:26.660451 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:48:26.664098 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:26.664076 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:48:42.642756 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:42.642712 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dh4xv"] Jul 06 17:48:42.643246 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:42.642942 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-dh4xv" podUID="16a07cf5-b126-4b65-8026-aefc9a946ebb" containerName="authorino" containerID="cri-o://7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1" gracePeriod=30 Jul 06 17:48:42.907600 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:42.907574 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dh4xv" Jul 06 17:48:42.981662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:42.981609 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khngk\" (UniqueName: \"kubernetes.io/projected/16a07cf5-b126-4b65-8026-aefc9a946ebb-kube-api-access-khngk\") pod \"16a07cf5-b126-4b65-8026-aefc9a946ebb\" (UID: \"16a07cf5-b126-4b65-8026-aefc9a946ebb\") " Jul 06 17:48:42.983789 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:42.983759 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16a07cf5-b126-4b65-8026-aefc9a946ebb-kube-api-access-khngk" (OuterVolumeSpecName: "kube-api-access-khngk") pod "16a07cf5-b126-4b65-8026-aefc9a946ebb" (UID: "16a07cf5-b126-4b65-8026-aefc9a946ebb"). InnerVolumeSpecName "kube-api-access-khngk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:48:43.083000 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.082963 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khngk\" (UniqueName: \"kubernetes.io/projected/16a07cf5-b126-4b65-8026-aefc9a946ebb-kube-api-access-khngk\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:48:43.564090 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.564051 2580 generic.go:356] "Generic (PLEG): container finished" podID="16a07cf5-b126-4b65-8026-aefc9a946ebb" containerID="7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1" exitCode=0 Jul 06 17:48:43.564090 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.564091 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dh4xv" Jul 06 17:48:43.564315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.564138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dh4xv" event={"ID":"16a07cf5-b126-4b65-8026-aefc9a946ebb","Type":"ContainerDied","Data":"7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1"} Jul 06 17:48:43.564315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.564180 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dh4xv" event={"ID":"16a07cf5-b126-4b65-8026-aefc9a946ebb","Type":"ContainerDied","Data":"11545a50f34b71019de16ac8e0842487a82646de37a03806c8feb8d9433f9928"} Jul 06 17:48:43.564315 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.564198 2580 scope.go:117] "RemoveContainer" containerID="7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1" Jul 06 17:48:43.572778 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.572758 2580 scope.go:117] "RemoveContainer" containerID="7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1" Jul 06 17:48:43.573037 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:48:43.573011 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1\": container with ID starting with 7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1 not found: ID does not exist" containerID="7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1" Jul 06 17:48:43.573081 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.573048 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1"} err="failed to get container status \"7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1\": rpc error: code = NotFound desc = could not find container \"7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1\": container with ID starting with 7fcb563520efea6509a0fb772c05d1ab267925e6df62a859dfeed8ca511bc8f1 not found: ID does not exist" Jul 06 17:48:43.584996 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.584970 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dh4xv"] Jul 06 17:48:43.590551 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:43.590531 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dh4xv"] Jul 06 17:48:44.767594 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:44.767561 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16a07cf5-b126-4b65-8026-aefc9a946ebb" path="/var/lib/kubelet/pods/16a07cf5-b126-4b65-8026-aefc9a946ebb/volumes" Jul 06 17:48:46.955356 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.955323 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-778c7c5c85-d7f4h"] Jul 06 17:48:46.955725 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.955706 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16a07cf5-b126-4b65-8026-aefc9a946ebb" containerName="authorino" Jul 06 17:48:46.955725 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.955718 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a07cf5-b126-4b65-8026-aefc9a946ebb" containerName="authorino" Jul 06 17:48:46.955809 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.955797 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="16a07cf5-b126-4b65-8026-aefc9a946ebb" containerName="authorino" Jul 06 17:48:46.962064 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.962043 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:46.965061 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.965035 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 17:48:46.965271 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.965253 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 17:48:46.966375 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.966344 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-k8rqh\"" Jul 06 17:48:46.967513 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:46.967491 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-778c7c5c85-d7f4h"] Jul 06 17:48:47.120539 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.120453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsmwv\" (UniqueName: \"kubernetes.io/projected/52b895a7-73e7-4933-84b7-1a437272c480-kube-api-access-bsmwv\") pod \"maas-controller-778c7c5c85-d7f4h\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:47.120539 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.120516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b895a7-73e7-4933-84b7-1a437272c480-webhook-cert\") pod \"maas-controller-778c7c5c85-d7f4h\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:47.131010 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.130964 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75789b675b-kq89k"] Jul 06 17:48:47.138078 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.138055 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.143516 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.143489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75789b675b-kq89k"] Jul 06 17:48:47.221808 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.221718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b895a7-73e7-4933-84b7-1a437272c480-webhook-cert\") pod \"maas-controller-778c7c5c85-d7f4h\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:47.221808 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.221805 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tz5m\" (UniqueName: \"kubernetes.io/projected/0b1a9a71-167c-4103-9aff-443acbdbba31-kube-api-access-7tz5m\") pod \"maas-controller-75789b675b-kq89k\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.222010 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.221838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsmwv\" (UniqueName: \"kubernetes.io/projected/52b895a7-73e7-4933-84b7-1a437272c480-kube-api-access-bsmwv\") pod \"maas-controller-778c7c5c85-d7f4h\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:47.222010 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.221885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1a9a71-167c-4103-9aff-443acbdbba31-webhook-cert\") pod \"maas-controller-75789b675b-kq89k\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.224145 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.224125 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b895a7-73e7-4933-84b7-1a437272c480-webhook-cert\") pod \"maas-controller-778c7c5c85-d7f4h\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:47.229793 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.229766 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsmwv\" (UniqueName: \"kubernetes.io/projected/52b895a7-73e7-4933-84b7-1a437272c480-kube-api-access-bsmwv\") pod \"maas-controller-778c7c5c85-d7f4h\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:47.274600 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.274570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:47.322714 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.322668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1a9a71-167c-4103-9aff-443acbdbba31-webhook-cert\") pod \"maas-controller-75789b675b-kq89k\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.322908 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.322805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tz5m\" (UniqueName: \"kubernetes.io/projected/0b1a9a71-167c-4103-9aff-443acbdbba31-kube-api-access-7tz5m\") pod \"maas-controller-75789b675b-kq89k\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.326560 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.325728 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1a9a71-167c-4103-9aff-443acbdbba31-webhook-cert\") pod \"maas-controller-75789b675b-kq89k\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.332059 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.332000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tz5m\" (UniqueName: \"kubernetes.io/projected/0b1a9a71-167c-4103-9aff-443acbdbba31-kube-api-access-7tz5m\") pod \"maas-controller-75789b675b-kq89k\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.409577 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.409549 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-778c7c5c85-d7f4h"] Jul 06 17:48:47.411117 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:48:47.411093 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b895a7_73e7_4933_84b7_1a437272c480.slice/crio-8af1da606b832d104896921941cc8f7d8308337a51cecc67d28f0acabcef32b2 WatchSource:0}: Error finding container 8af1da606b832d104896921941cc8f7d8308337a51cecc67d28f0acabcef32b2: Status 404 returned error can't find the container with id 8af1da606b832d104896921941cc8f7d8308337a51cecc67d28f0acabcef32b2 Jul 06 17:48:47.450250 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.450212 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:47.587755 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.587712 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" event={"ID":"52b895a7-73e7-4933-84b7-1a437272c480","Type":"ContainerStarted","Data":"8af1da606b832d104896921941cc8f7d8308337a51cecc67d28f0acabcef32b2"} Jul 06 17:48:47.616141 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:47.616116 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75789b675b-kq89k"] Jul 06 17:48:47.618302 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:48:47.618262 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1a9a71_167c_4103_9aff_443acbdbba31.slice/crio-44af8e0fdab017d8aa9f6d9d27bad3f35dc94697a71d81006e34c21e2cfcd238 WatchSource:0}: Error finding container 44af8e0fdab017d8aa9f6d9d27bad3f35dc94697a71d81006e34c21e2cfcd238: Status 404 returned error can't find the container with id 44af8e0fdab017d8aa9f6d9d27bad3f35dc94697a71d81006e34c21e2cfcd238 Jul 06 17:48:48.599687 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:48.599178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75789b675b-kq89k" event={"ID":"0b1a9a71-167c-4103-9aff-443acbdbba31","Type":"ContainerStarted","Data":"44af8e0fdab017d8aa9f6d9d27bad3f35dc94697a71d81006e34c21e2cfcd238"} Jul 06 17:48:50.609703 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:50.609668 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" event={"ID":"52b895a7-73e7-4933-84b7-1a437272c480","Type":"ContainerStarted","Data":"954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c"} Jul 06 17:48:50.610195 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:50.609754 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:48:50.611139 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:50.611112 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75789b675b-kq89k" event={"ID":"0b1a9a71-167c-4103-9aff-443acbdbba31","Type":"ContainerStarted","Data":"2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d"} Jul 06 17:48:50.611232 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:50.611193 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:48:50.628459 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:50.628408 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" podStartSLOduration=1.8288587490000001 podStartE2EDuration="4.628392137s" totalImagesPullingTime="2.799533388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:48:50.627900696 +0000 UTC m=+624.378268288" watchObservedRunningTime="2026-07-06 17:48:50.628392137 +0000 UTC m=+624.378759712" Jul 06 17:48:50.646804 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:48:50.646751 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75789b675b-kq89k" podStartSLOduration=1.047792149 podStartE2EDuration="3.646733331s" totalImagesPullingTime="2.598941182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:48:50.646223567 +0000 UTC m=+624.396591145" watchObservedRunningTime="2026-07-06 17:48:50.646733331 +0000 UTC m=+624.397100965" Jul 06 17:49:01.622103 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:01.622051 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:49:01.622726 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:01.622154 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:49:01.672228 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:01.672192 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-778c7c5c85-d7f4h"] Jul 06 17:49:01.672505 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:01.672450 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" podUID="52b895a7-73e7-4933-84b7-1a437272c480" containerName="manager" containerID="cri-o://954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c" gracePeriod=30 Jul 06 17:49:01.922232 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:01.922204 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:49:02.064760 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.064726 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsmwv\" (UniqueName: \"kubernetes.io/projected/52b895a7-73e7-4933-84b7-1a437272c480-kube-api-access-bsmwv\") pod \"52b895a7-73e7-4933-84b7-1a437272c480\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " Jul 06 17:49:02.064938 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.064798 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b895a7-73e7-4933-84b7-1a437272c480-webhook-cert\") pod \"52b895a7-73e7-4933-84b7-1a437272c480\" (UID: \"52b895a7-73e7-4933-84b7-1a437272c480\") " Jul 06 17:49:02.067023 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.066993 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52b895a7-73e7-4933-84b7-1a437272c480-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "52b895a7-73e7-4933-84b7-1a437272c480" (UID: "52b895a7-73e7-4933-84b7-1a437272c480"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:49:02.067023 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.066995 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b895a7-73e7-4933-84b7-1a437272c480-kube-api-access-bsmwv" (OuterVolumeSpecName: "kube-api-access-bsmwv") pod "52b895a7-73e7-4933-84b7-1a437272c480" (UID: "52b895a7-73e7-4933-84b7-1a437272c480"). InnerVolumeSpecName "kube-api-access-bsmwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:49:02.166391 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.166304 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52b895a7-73e7-4933-84b7-1a437272c480-webhook-cert\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:49:02.166391 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.166336 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsmwv\" (UniqueName: \"kubernetes.io/projected/52b895a7-73e7-4933-84b7-1a437272c480-kube-api-access-bsmwv\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:49:02.668280 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.668243 2580 generic.go:356] "Generic (PLEG): container finished" podID="52b895a7-73e7-4933-84b7-1a437272c480" containerID="954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c" exitCode=0 Jul 06 17:49:02.668770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.668309 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" Jul 06 17:49:02.668770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.668335 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" event={"ID":"52b895a7-73e7-4933-84b7-1a437272c480","Type":"ContainerDied","Data":"954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c"} Jul 06 17:49:02.668770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.668400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778c7c5c85-d7f4h" event={"ID":"52b895a7-73e7-4933-84b7-1a437272c480","Type":"ContainerDied","Data":"8af1da606b832d104896921941cc8f7d8308337a51cecc67d28f0acabcef32b2"} Jul 06 17:49:02.668770 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.668432 2580 scope.go:117] "RemoveContainer" containerID="954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c" Jul 06 17:49:02.677829 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.677639 2580 scope.go:117] "RemoveContainer" containerID="954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c" Jul 06 17:49:02.677939 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:49:02.677917 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c\": container with ID starting with 954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c not found: ID does not exist" containerID="954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c" Jul 06 17:49:02.677982 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.677949 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c"} err="failed to get container status \"954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c\": rpc error: code = NotFound desc = could not find container \"954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c\": container with ID starting with 954ed9a2610d60d731e185e5b9160e72ea33faf70abc2dffcb5e99fcb2548d8c not found: ID does not exist" Jul 06 17:49:02.690121 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.690093 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-778c7c5c85-d7f4h"] Jul 06 17:49:02.693696 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.693670 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-778c7c5c85-d7f4h"] Jul 06 17:49:02.767582 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:02.767552 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52b895a7-73e7-4933-84b7-1a437272c480" path="/var/lib/kubelet/pods/52b895a7-73e7-4933-84b7-1a437272c480/volumes" Jul 06 17:49:22.962079 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.962041 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7cf57666c-x2md2"] Jul 06 17:49:22.964452 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.962434 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52b895a7-73e7-4933-84b7-1a437272c480" containerName="manager" Jul 06 17:49:22.964452 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.962445 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="52b895a7-73e7-4933-84b7-1a437272c480" containerName="manager" Jul 06 17:49:22.964452 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.962501 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="52b895a7-73e7-4933-84b7-1a437272c480" containerName="manager" Jul 06 17:49:22.965378 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.965355 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:22.968621 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.968597 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 17:49:22.968756 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.968601 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-z2cpm\"" Jul 06 17:49:22.976953 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:22.976930 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7cf57666c-x2md2"] Jul 06 17:49:23.038051 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.038002 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-openshift-service-ca\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.038236 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.038118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-maas-api-tls\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.038236 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.038191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lblr\" (UniqueName: \"kubernetes.io/projected/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-kube-api-access-5lblr\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.138999 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.138959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-maas-api-tls\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.139172 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.139029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lblr\" (UniqueName: \"kubernetes.io/projected/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-kube-api-access-5lblr\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.139172 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.139070 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-openshift-service-ca\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.139699 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.139682 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-openshift-service-ca\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.141527 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.141504 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-maas-api-tls\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.148084 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.148054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lblr\" (UniqueName: \"kubernetes.io/projected/02e8ecaa-fd1a-4b54-a03d-f1baf16774ae-kube-api-access-5lblr\") pod \"maas-api-7cf57666c-x2md2\" (UID: \"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae\") " pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.276030 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.275994 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:23.423248 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.423224 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7cf57666c-x2md2"] Jul 06 17:49:23.425172 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:49:23.425141 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e8ecaa_fd1a_4b54_a03d_f1baf16774ae.slice/crio-9b6745b41044234bea4a1705466881e06878c809635dd6185768f9abf0805f86 WatchSource:0}: Error finding container 9b6745b41044234bea4a1705466881e06878c809635dd6185768f9abf0805f86: Status 404 returned error can't find the container with id 9b6745b41044234bea4a1705466881e06878c809635dd6185768f9abf0805f86 Jul 06 17:49:23.462995 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.462962 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-s5dxt"] Jul 06 17:49:23.467453 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.467434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.470510 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.470486 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rqk58\"" Jul 06 17:49:23.470630 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.470531 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 17:49:23.475279 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.475256 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-s5dxt"] Jul 06 17:49:23.543695 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.543607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b859z\" (UniqueName: \"kubernetes.io/projected/72aee57d-e8f8-49f0-b294-d5750e1081c0-kube-api-access-b859z\") pod \"payload-pre-processing-7964457696-s5dxt\" (UID: \"72aee57d-e8f8-49f0-b294-d5750e1081c0\") " pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.543836 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.543741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72aee57d-e8f8-49f0-b294-d5750e1081c0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-s5dxt\" (UID: \"72aee57d-e8f8-49f0-b294-d5750e1081c0\") " pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.644561 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.644520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72aee57d-e8f8-49f0-b294-d5750e1081c0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-s5dxt\" (UID: \"72aee57d-e8f8-49f0-b294-d5750e1081c0\") " pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.644739 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.644587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b859z\" (UniqueName: \"kubernetes.io/projected/72aee57d-e8f8-49f0-b294-d5750e1081c0-kube-api-access-b859z\") pod \"payload-pre-processing-7964457696-s5dxt\" (UID: \"72aee57d-e8f8-49f0-b294-d5750e1081c0\") " pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.645242 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.645224 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72aee57d-e8f8-49f0-b294-d5750e1081c0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-s5dxt\" (UID: \"72aee57d-e8f8-49f0-b294-d5750e1081c0\") " pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.652571 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.652544 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b859z\" (UniqueName: \"kubernetes.io/projected/72aee57d-e8f8-49f0-b294-d5750e1081c0-kube-api-access-b859z\") pod \"payload-pre-processing-7964457696-s5dxt\" (UID: \"72aee57d-e8f8-49f0-b294-d5750e1081c0\") " pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.770229 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.770189 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7cf57666c-x2md2" event={"ID":"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae","Type":"ContainerStarted","Data":"9b6745b41044234bea4a1705466881e06878c809635dd6185768f9abf0805f86"} Jul 06 17:49:23.779421 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.779399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:23.908406 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:23.908381 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-s5dxt"] Jul 06 17:49:23.910004 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:49:23.909977 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72aee57d_e8f8_49f0_b294_d5750e1081c0.slice/crio-df5ff63a26f0c72d37de408eb98a2d962021b30c866822e38203df06a0d3174a WatchSource:0}: Error finding container df5ff63a26f0c72d37de408eb98a2d962021b30c866822e38203df06a0d3174a: Status 404 returned error can't find the container with id df5ff63a26f0c72d37de408eb98a2d962021b30c866822e38203df06a0d3174a Jul 06 17:49:24.776713 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:24.776673 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" event={"ID":"72aee57d-e8f8-49f0-b294-d5750e1081c0","Type":"ContainerStarted","Data":"df5ff63a26f0c72d37de408eb98a2d962021b30c866822e38203df06a0d3174a"} Jul 06 17:49:25.782145 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:25.782101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7cf57666c-x2md2" event={"ID":"02e8ecaa-fd1a-4b54-a03d-f1baf16774ae","Type":"ContainerStarted","Data":"670fcce7affa250c43facb4a687d868975164fdab2ec27c085668db0f0db6655"} Jul 06 17:49:25.782619 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:25.782203 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:25.783565 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:25.783544 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" event={"ID":"72aee57d-e8f8-49f0-b294-d5750e1081c0","Type":"ContainerStarted","Data":"f06b0b37c4262c90e6324b7d6fc69a19e6eb6604f61f28b689d8dd1e35ff4d88"} Jul 06 17:49:25.783689 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:25.783669 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:49:25.800939 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:25.800888 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-7cf57666c-x2md2" podStartSLOduration=1.656093306 podStartE2EDuration="3.800873865s" totalImagesPullingTime="2.144780559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:25.799305979 +0000 UTC m=+659.549673550" watchObservedRunningTime="2026-07-06 17:49:25.800873865 +0000 UTC m=+659.551241440" Jul 06 17:49:25.818408 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:25.818367 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" podStartSLOduration=1.156740353 podStartE2EDuration="2.818352811s" totalImagesPullingTime="1.661612458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:25.817695358 +0000 UTC m=+659.568062957" watchObservedRunningTime="2026-07-06 17:49:25.818352811 +0000 UTC m=+659.568720387" Jul 06 17:49:31.793099 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:31.793066 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-7cf57666c-x2md2" Jul 06 17:49:36.789679 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:49:36.789650 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-s5dxt" Jul 06 17:50:10.091658 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.091623 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk"] Jul 06 17:50:10.096750 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.096730 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.099481 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.099459 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 17:50:10.101807 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.101736 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 17:50:10.101807 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.101771 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 17:50:10.102014 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.101876 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hkwh9\"" Jul 06 17:50:10.104691 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.104489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk"] Jul 06 17:50:10.171404 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.171353 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.171619 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.171473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf7vf\" (UniqueName: \"kubernetes.io/projected/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-kube-api-access-jf7vf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.171619 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.171536 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.171747 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.171652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.171747 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.171703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.171871 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.171767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273041 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273041 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273352 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf7vf\" (UniqueName: \"kubernetes.io/projected/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-kube-api-access-jf7vf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273352 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273352 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273550 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273550 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273518 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273662 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273632 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.273712 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.273686 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.275477 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.275454 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.275795 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.275777 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.280602 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.280582 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf7vf\" (UniqueName: \"kubernetes.io/projected/d8661a91-f5b6-42ca-9f89-af1d9ba7fc33-kube-api-access-jf7vf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-l99xk\" (UID: \"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.408267 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.408181 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:10.545057 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.545024 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk"] Jul 06 17:50:10.547828 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:50:10.547801 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8661a91_f5b6_42ca_9f89_af1d9ba7fc33.slice/crio-75b4def5786c8bf8abfa07921910bf8ea48e97709d93717d15d120c6a08b1078 WatchSource:0}: Error finding container 75b4def5786c8bf8abfa07921910bf8ea48e97709d93717d15d120c6a08b1078: Status 404 returned error can't find the container with id 75b4def5786c8bf8abfa07921910bf8ea48e97709d93717d15d120c6a08b1078 Jul 06 17:50:10.549239 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.549219 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 17:50:10.994524 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:10.994486 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" event={"ID":"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33","Type":"ContainerStarted","Data":"75b4def5786c8bf8abfa07921910bf8ea48e97709d93717d15d120c6a08b1078"} Jul 06 17:50:12.001368 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:12.001334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" event={"ID":"d8661a91-f5b6-42ca-9f89-af1d9ba7fc33","Type":"ContainerStarted","Data":"87c5347e4d9b5ae600e23c3f2a06359f7ecf27353eea5be5ef1b795951011a23"} Jul 06 17:50:12.023151 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:12.023100 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" podStartSLOduration=1.009344787 podStartE2EDuration="2.023084207s" totalImagesPullingTime="1.01373942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:50:12.022771497 +0000 UTC m=+705.773139089" watchObservedRunningTime="2026-07-06 17:50:12.023084207 +0000 UTC m=+705.773451782" Jul 06 17:50:20.409173 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:20.409081 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:20.409173 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:20.409171 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:20.421713 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:20.421687 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:50:21.056955 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:50:21.056926 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-l99xk" Jul 06 17:51:41.446283 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.446249 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55cbc7ddf6-9vtj5"] Jul 06 17:51:41.449917 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.449896 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.454088 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.454048 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 17:51:41.454231 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.454052 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-294km\"" Jul 06 17:51:41.459417 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.459390 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cbc7ddf6-9vtj5"] Jul 06 17:51:41.609410 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.609362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5zm6\" (UniqueName: \"kubernetes.io/projected/90b1f7ea-c93d-405a-95e9-19ad67eb63eb-kube-api-access-j5zm6\") pod \"authorino-55cbc7ddf6-9vtj5\" (UID: \"90b1f7ea-c93d-405a-95e9-19ad67eb63eb\") " pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.609601 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.609450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b1f7ea-c93d-405a-95e9-19ad67eb63eb-tls-cert\") pod \"authorino-55cbc7ddf6-9vtj5\" (UID: \"90b1f7ea-c93d-405a-95e9-19ad67eb63eb\") " pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.710243 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.710140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5zm6\" (UniqueName: \"kubernetes.io/projected/90b1f7ea-c93d-405a-95e9-19ad67eb63eb-kube-api-access-j5zm6\") pod \"authorino-55cbc7ddf6-9vtj5\" (UID: \"90b1f7ea-c93d-405a-95e9-19ad67eb63eb\") " pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.710243 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.710223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b1f7ea-c93d-405a-95e9-19ad67eb63eb-tls-cert\") pod \"authorino-55cbc7ddf6-9vtj5\" (UID: \"90b1f7ea-c93d-405a-95e9-19ad67eb63eb\") " pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.712815 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.712785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b1f7ea-c93d-405a-95e9-19ad67eb63eb-tls-cert\") pod \"authorino-55cbc7ddf6-9vtj5\" (UID: \"90b1f7ea-c93d-405a-95e9-19ad67eb63eb\") " pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.718671 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.718646 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5zm6\" (UniqueName: \"kubernetes.io/projected/90b1f7ea-c93d-405a-95e9-19ad67eb63eb-kube-api-access-j5zm6\") pod \"authorino-55cbc7ddf6-9vtj5\" (UID: \"90b1f7ea-c93d-405a-95e9-19ad67eb63eb\") " pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.765655 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.765621 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" Jul 06 17:51:41.901471 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:41.901445 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cbc7ddf6-9vtj5"] Jul 06 17:51:41.905644 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:51:41.905614 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90b1f7ea_c93d_405a_95e9_19ad67eb63eb.slice/crio-12070ed17ae8fd6eedcb38cc436255ef031b12bc7acd59f24a8b01a46629df5c WatchSource:0}: Error finding container 12070ed17ae8fd6eedcb38cc436255ef031b12bc7acd59f24a8b01a46629df5c: Status 404 returned error can't find the container with id 12070ed17ae8fd6eedcb38cc436255ef031b12bc7acd59f24a8b01a46629df5c Jul 06 17:51:42.426045 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:42.426007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" event={"ID":"90b1f7ea-c93d-405a-95e9-19ad67eb63eb","Type":"ContainerStarted","Data":"12070ed17ae8fd6eedcb38cc436255ef031b12bc7acd59f24a8b01a46629df5c"} Jul 06 17:51:43.434844 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:43.434803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" event={"ID":"90b1f7ea-c93d-405a-95e9-19ad67eb63eb","Type":"ContainerStarted","Data":"511c875d704d883e39993f1edc380ff060f23a78faa4ce57d5fc19a7bd0e9448"} Jul 06 17:51:43.451531 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:51:43.451477 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55cbc7ddf6-9vtj5" podStartSLOduration=1.850063264 podStartE2EDuration="2.451459699s" totalImagesPullingTime="601.396435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:43.45031804 +0000 UTC m=+797.200685633" watchObservedRunningTime="2026-07-06 17:51:43.451459699 +0000 UTC m=+797.201827296" Jul 06 17:52:56.358545 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.358508 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75789b675b-kq89k"] Jul 06 17:52:56.359054 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.358756 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-75789b675b-kq89k" podUID="0b1a9a71-167c-4103-9aff-443acbdbba31" containerName="manager" containerID="cri-o://2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d" gracePeriod=30 Jul 06 17:52:56.615174 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.615114 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:52:56.713517 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.713484 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tz5m\" (UniqueName: \"kubernetes.io/projected/0b1a9a71-167c-4103-9aff-443acbdbba31-kube-api-access-7tz5m\") pod \"0b1a9a71-167c-4103-9aff-443acbdbba31\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " Jul 06 17:52:56.713718 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.713527 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1a9a71-167c-4103-9aff-443acbdbba31-webhook-cert\") pod \"0b1a9a71-167c-4103-9aff-443acbdbba31\" (UID: \"0b1a9a71-167c-4103-9aff-443acbdbba31\") " Jul 06 17:52:56.715706 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.715656 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b1a9a71-167c-4103-9aff-443acbdbba31-kube-api-access-7tz5m" (OuterVolumeSpecName: "kube-api-access-7tz5m") pod "0b1a9a71-167c-4103-9aff-443acbdbba31" (UID: "0b1a9a71-167c-4103-9aff-443acbdbba31"). InnerVolumeSpecName "kube-api-access-7tz5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:52:56.715706 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.715700 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1a9a71-167c-4103-9aff-443acbdbba31-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0b1a9a71-167c-4103-9aff-443acbdbba31" (UID: "0b1a9a71-167c-4103-9aff-443acbdbba31"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:52:56.792033 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.791997 2580 generic.go:356] "Generic (PLEG): container finished" podID="0b1a9a71-167c-4103-9aff-443acbdbba31" containerID="2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d" exitCode=0 Jul 06 17:52:56.792198 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.792039 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75789b675b-kq89k" Jul 06 17:52:56.792198 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.792074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75789b675b-kq89k" event={"ID":"0b1a9a71-167c-4103-9aff-443acbdbba31","Type":"ContainerDied","Data":"2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d"} Jul 06 17:52:56.792198 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.792125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75789b675b-kq89k" event={"ID":"0b1a9a71-167c-4103-9aff-443acbdbba31","Type":"ContainerDied","Data":"44af8e0fdab017d8aa9f6d9d27bad3f35dc94697a71d81006e34c21e2cfcd238"} Jul 06 17:52:56.792198 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.792154 2580 scope.go:117] "RemoveContainer" containerID="2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d" Jul 06 17:52:56.801101 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.801083 2580 scope.go:117] "RemoveContainer" containerID="2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d" Jul 06 17:52:56.801371 ip-10-0-138-108 kubenswrapper[2580]: E0706 17:52:56.801350 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d\": container with ID starting with 2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d not found: ID does not exist" containerID="2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d" Jul 06 17:52:56.801430 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.801378 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d"} err="failed to get container status \"2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d\": rpc error: code = NotFound desc = could not find container \"2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d\": container with ID starting with 2a7d328dd155a6ab59d1dd763667704e299aa7f21eec5d2c0afe3654205f445d not found: ID does not exist" Jul 06 17:52:56.809595 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.809570 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75789b675b-kq89k"] Jul 06 17:52:56.813152 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.813132 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75789b675b-kq89k"] Jul 06 17:52:56.814758 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.814740 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tz5m\" (UniqueName: \"kubernetes.io/projected/0b1a9a71-167c-4103-9aff-443acbdbba31-kube-api-access-7tz5m\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:52:56.814820 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:56.814764 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b1a9a71-167c-4103-9aff-443acbdbba31-webhook-cert\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:52:57.610326 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.610267 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75789b675b-f7tk2"] Jul 06 17:52:57.610747 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.610675 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b1a9a71-167c-4103-9aff-443acbdbba31" containerName="manager" Jul 06 17:52:57.610747 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.610686 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1a9a71-167c-4103-9aff-443acbdbba31" containerName="manager" Jul 06 17:52:57.610747 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.610747 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b1a9a71-167c-4103-9aff-443acbdbba31" containerName="manager" Jul 06 17:52:57.615069 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.615051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:57.619541 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.619520 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 17:52:57.619541 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.619529 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 17:52:57.619735 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.619536 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-k8rqh\"" Jul 06 17:52:57.624858 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.624832 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75789b675b-f7tk2"] Jul 06 17:52:57.723142 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.723108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1531bbb3-ff39-4521-9990-924bb99da441-webhook-cert\") pod \"maas-controller-75789b675b-f7tk2\" (UID: \"1531bbb3-ff39-4521-9990-924bb99da441\") " pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:57.723356 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.723180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx4bb\" (UniqueName: \"kubernetes.io/projected/1531bbb3-ff39-4521-9990-924bb99da441-kube-api-access-cx4bb\") pod \"maas-controller-75789b675b-f7tk2\" (UID: \"1531bbb3-ff39-4521-9990-924bb99da441\") " pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:57.824464 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.824429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx4bb\" (UniqueName: \"kubernetes.io/projected/1531bbb3-ff39-4521-9990-924bb99da441-kube-api-access-cx4bb\") pod \"maas-controller-75789b675b-f7tk2\" (UID: \"1531bbb3-ff39-4521-9990-924bb99da441\") " pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:57.824680 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.824553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1531bbb3-ff39-4521-9990-924bb99da441-webhook-cert\") pod \"maas-controller-75789b675b-f7tk2\" (UID: \"1531bbb3-ff39-4521-9990-924bb99da441\") " pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:57.827073 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.827050 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1531bbb3-ff39-4521-9990-924bb99da441-webhook-cert\") pod \"maas-controller-75789b675b-f7tk2\" (UID: \"1531bbb3-ff39-4521-9990-924bb99da441\") " pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:57.833631 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.833603 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx4bb\" (UniqueName: \"kubernetes.io/projected/1531bbb3-ff39-4521-9990-924bb99da441-kube-api-access-cx4bb\") pod \"maas-controller-75789b675b-f7tk2\" (UID: \"1531bbb3-ff39-4521-9990-924bb99da441\") " pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:57.926681 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:57.926579 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:58.062241 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:58.062213 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75789b675b-f7tk2"] Jul 06 17:52:58.065115 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:52:58.065085 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1531bbb3_ff39_4521_9990_924bb99da441.slice/crio-da24d60bd84579357877411ad64f9811ed86e957ec0434859392454a3f26d775 WatchSource:0}: Error finding container da24d60bd84579357877411ad64f9811ed86e957ec0434859392454a3f26d775: Status 404 returned error can't find the container with id da24d60bd84579357877411ad64f9811ed86e957ec0434859392454a3f26d775 Jul 06 17:52:58.768098 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:58.768066 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b1a9a71-167c-4103-9aff-443acbdbba31" path="/var/lib/kubelet/pods/0b1a9a71-167c-4103-9aff-443acbdbba31/volumes" Jul 06 17:52:58.804105 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:58.804013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75789b675b-f7tk2" event={"ID":"1531bbb3-ff39-4521-9990-924bb99da441","Type":"ContainerStarted","Data":"ae6e2784887b0f4081091ab71bf4a23b887dd596fa387ef37279d040aa34d38f"} Jul 06 17:52:58.804105 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:58.804063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75789b675b-f7tk2" event={"ID":"1531bbb3-ff39-4521-9990-924bb99da441","Type":"ContainerStarted","Data":"da24d60bd84579357877411ad64f9811ed86e957ec0434859392454a3f26d775"} Jul 06 17:52:58.804331 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:58.804125 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:52:58.821547 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:52:58.821492 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75789b675b-f7tk2" podStartSLOduration=1.354892348 podStartE2EDuration="1.821471921s" totalImagesPullingTime="466.579573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:52:58.81958686 +0000 UTC m=+872.569954438" watchObservedRunningTime="2026-07-06 17:52:58.821471921 +0000 UTC m=+872.571839515" Jul 06 17:53:09.818039 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:53:09.818007 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75789b675b-f7tk2" Jul 06 17:53:26.697817 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:53:26.697734 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:53:26.697817 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:53:26.697783 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:57:43.386960 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:43.386918 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8"] Jul 06 17:57:43.387588 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:43.387236 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" podUID="bb21d8f6-8707-4649-8ee4-a88da16bbb7d" containerName="manager" containerID="cri-o://5e00dc36a5365a9d8c632d9b836dccbfff38cac002b1bab768228991e774c183" gracePeriod=10 Jul 06 17:57:45.210683 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.210652 2580 generic.go:356] "Generic (PLEG): container finished" podID="bb21d8f6-8707-4649-8ee4-a88da16bbb7d" containerID="5e00dc36a5365a9d8c632d9b836dccbfff38cac002b1bab768228991e774c183" exitCode=0 Jul 06 17:57:45.211031 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.210725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" event={"ID":"bb21d8f6-8707-4649-8ee4-a88da16bbb7d","Type":"ContainerDied","Data":"5e00dc36a5365a9d8c632d9b836dccbfff38cac002b1bab768228991e774c183"} Jul 06 17:57:45.343602 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.343574 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:57:45.413566 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.413531 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-extensions-socket-volume\") pod \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " Jul 06 17:57:45.413738 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.413588 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfscf\" (UniqueName: \"kubernetes.io/projected/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-kube-api-access-qfscf\") pod \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\" (UID: \"bb21d8f6-8707-4649-8ee4-a88da16bbb7d\") " Jul 06 17:57:45.413912 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.413887 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bb21d8f6-8707-4649-8ee4-a88da16bbb7d" (UID: "bb21d8f6-8707-4649-8ee4-a88da16bbb7d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:57:45.415648 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.415622 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-kube-api-access-qfscf" (OuterVolumeSpecName: "kube-api-access-qfscf") pod "bb21d8f6-8707-4649-8ee4-a88da16bbb7d" (UID: "bb21d8f6-8707-4649-8ee4-a88da16bbb7d"). InnerVolumeSpecName "kube-api-access-qfscf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:57:45.514611 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.514575 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-extensions-socket-volume\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:57:45.514611 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:45.514605 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfscf\" (UniqueName: \"kubernetes.io/projected/bb21d8f6-8707-4649-8ee4-a88da16bbb7d-kube-api-access-qfscf\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 17:57:46.216350 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:46.216315 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" Jul 06 17:57:46.216350 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:46.216325 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8" event={"ID":"bb21d8f6-8707-4649-8ee4-a88da16bbb7d","Type":"ContainerDied","Data":"9afc0a9c76a69a69daf5f25bd0a00cb9be4919ac9d62b1ae856a8778489f1f29"} Jul 06 17:57:46.216889 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:46.216399 2580 scope.go:117] "RemoveContainer" containerID="5e00dc36a5365a9d8c632d9b836dccbfff38cac002b1bab768228991e774c183" Jul 06 17:57:46.238769 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:46.238743 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8"] Jul 06 17:57:46.244258 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:46.244229 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gnts8"] Jul 06 17:57:46.768424 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:57:46.768388 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb21d8f6-8707-4649-8ee4-a88da16bbb7d" path="/var/lib/kubelet/pods/bb21d8f6-8707-4649-8ee4-a88da16bbb7d/volumes" Jul 06 17:58:26.726143 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:26.726109 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:58:26.727955 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:26.727934 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 17:58:44.559930 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.559891 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt"] Jul 06 17:58:44.560328 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.560268 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb21d8f6-8707-4649-8ee4-a88da16bbb7d" containerName="manager" Jul 06 17:58:44.560328 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.560281 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb21d8f6-8707-4649-8ee4-a88da16bbb7d" containerName="manager" Jul 06 17:58:44.560404 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.560384 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb21d8f6-8707-4649-8ee4-a88da16bbb7d" containerName="manager" Jul 06 17:58:44.564597 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.564581 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:44.567579 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.567556 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-82b8x\"" Jul 06 17:58:44.576923 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.576899 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt"] Jul 06 17:58:44.671481 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.671443 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hpp4\" (UniqueName: \"kubernetes.io/projected/5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b-kube-api-access-2hpp4\") pod \"kuadrant-operator-controller-manager-86878c769f-q2mdt\" (UID: \"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:44.671658 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.671511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q2mdt\" (UID: \"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:44.772706 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.772675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hpp4\" (UniqueName: \"kubernetes.io/projected/5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b-kube-api-access-2hpp4\") pod \"kuadrant-operator-controller-manager-86878c769f-q2mdt\" (UID: \"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:44.772871 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.772727 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q2mdt\" (UID: \"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:44.773088 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.773071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q2mdt\" (UID: \"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:44.785013 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.784979 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hpp4\" (UniqueName: \"kubernetes.io/projected/5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b-kube-api-access-2hpp4\") pod \"kuadrant-operator-controller-manager-86878c769f-q2mdt\" (UID: \"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:44.875102 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:44.875016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:45.019089 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:45.019058 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt"] Jul 06 17:58:45.021616 ip-10-0-138-108 kubenswrapper[2580]: W0706 17:58:45.021579 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9d6f6f_f0b4_4194_ae62_7fc4085dc18b.slice/crio-9bf18f2aa98f69979f110df27d3a9e6bb12f8dddb27f74a5c1c9f6fe1d595fbb WatchSource:0}: Error finding container 9bf18f2aa98f69979f110df27d3a9e6bb12f8dddb27f74a5c1c9f6fe1d595fbb: Status 404 returned error can't find the container with id 9bf18f2aa98f69979f110df27d3a9e6bb12f8dddb27f74a5c1c9f6fe1d595fbb Jul 06 17:58:45.497074 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:45.497035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" event={"ID":"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b","Type":"ContainerStarted","Data":"744bf54ca359f88c7616df5bf7cacc6107b608b3f4bd32ed6ca30e4642a8732e"} Jul 06 17:58:45.497245 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:45.497098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" event={"ID":"5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b","Type":"ContainerStarted","Data":"9bf18f2aa98f69979f110df27d3a9e6bb12f8dddb27f74a5c1c9f6fe1d595fbb"} Jul 06 17:58:45.497245 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:45.497139 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 17:58:45.515917 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:45.515866 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" podStartSLOduration=1.5158502139999999 podStartE2EDuration="1.515850214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:58:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:58:45.515275656 +0000 UTC m=+1219.265643233" watchObservedRunningTime="2026-07-06 17:58:45.515850214 +0000 UTC m=+1219.266217789" Jul 06 17:58:56.503686 ip-10-0-138-108 kubenswrapper[2580]: I0706 17:58:56.503647 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2mdt" Jul 06 18:03:26.757519 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:03:26.757491 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:03:26.758040 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:03:26.757873 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:05:08.102877 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.102842 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw"] Jul 06 18:05:08.106919 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.106897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.111027 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.111000 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-2d5b0cda-serving-cert\"" Jul 06 18:05:08.124834 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.124806 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw"] Jul 06 18:05:08.155086 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.152599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819ab857-6e46-4438-954a-d63331d501c3-openshift-service-ca\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.155086 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.152703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.155086 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.152762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8bdx\" (UniqueName: \"kubernetes.io/projected/819ab857-6e46-4438-954a-d63331d501c3-kube-api-access-k8bdx\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.253879 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.253835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.254063 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.253907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bdx\" (UniqueName: \"kubernetes.io/projected/819ab857-6e46-4438-954a-d63331d501c3-kube-api-access-k8bdx\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.254063 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.254020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819ab857-6e46-4438-954a-d63331d501c3-openshift-service-ca\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.254063 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:05:08.254023 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-2d5b0cda-serving-cert: secret "maas-api-e2e-ait-2d5b0cda-serving-cert" not found Jul 06 18:05:08.254219 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:05:08.254125 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls podName:819ab857-6e46-4438-954a-d63331d501c3 nodeName:}" failed. No retries permitted until 2026-07-06 18:05:08.754102624 +0000 UTC m=+1602.504470179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls") pod "maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" (UID: "819ab857-6e46-4438-954a-d63331d501c3") : secret "maas-api-e2e-ait-2d5b0cda-serving-cert" not found Jul 06 18:05:08.254700 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.254680 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819ab857-6e46-4438-954a-d63331d501c3-openshift-service-ca\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.268897 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.268861 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8bdx\" (UniqueName: \"kubernetes.io/projected/819ab857-6e46-4438-954a-d63331d501c3-kube-api-access-k8bdx\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.759455 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.759407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:08.762074 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:08.762047 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls\") pod \"maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:09.017486 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:09.017390 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:09.162344 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:09.162305 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw"] Jul 06 18:05:09.165923 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:05:09.165893 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819ab857_6e46_4438_954a_d63331d501c3.slice/crio-126f47c7c2a06a28da9bf3f2b09d0ccd7d219436bfeb2b06c2f903ba35c3a3a3 WatchSource:0}: Error finding container 126f47c7c2a06a28da9bf3f2b09d0ccd7d219436bfeb2b06c2f903ba35c3a3a3: Status 404 returned error can't find the container with id 126f47c7c2a06a28da9bf3f2b09d0ccd7d219436bfeb2b06c2f903ba35c3a3a3 Jul 06 18:05:09.167348 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:09.167327 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:05:09.391410 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:09.391321 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" event={"ID":"819ab857-6e46-4438-954a-d63331d501c3","Type":"ContainerStarted","Data":"126f47c7c2a06a28da9bf3f2b09d0ccd7d219436bfeb2b06c2f903ba35c3a3a3"} Jul 06 18:05:10.397198 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:10.397158 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" event={"ID":"819ab857-6e46-4438-954a-d63331d501c3","Type":"ContainerStarted","Data":"b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857"} Jul 06 18:05:10.399460 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:10.397270 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:10.414504 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:10.414458 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" podStartSLOduration=1.818852429 podStartE2EDuration="2.414440084s" totalImagesPullingTime="595.587655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:05:10.413665269 +0000 UTC m=+1604.164032846" watchObservedRunningTime="2026-07-06 18:05:10.414440084 +0000 UTC m=+1604.164807659" Jul 06 18:05:13.630174 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:13.630133 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw"] Jul 06 18:05:13.630649 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:13.630391 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" podUID="819ab857-6e46-4438-954a-d63331d501c3" containerName="maas-api" containerID="cri-o://b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857" gracePeriod=30 Jul 06 18:05:13.885866 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:13.885806 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:14.007194 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.007134 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819ab857-6e46-4438-954a-d63331d501c3-openshift-service-ca\") pod \"819ab857-6e46-4438-954a-d63331d501c3\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " Jul 06 18:05:14.007412 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.007223 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls\") pod \"819ab857-6e46-4438-954a-d63331d501c3\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " Jul 06 18:05:14.007412 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.007252 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8bdx\" (UniqueName: \"kubernetes.io/projected/819ab857-6e46-4438-954a-d63331d501c3-kube-api-access-k8bdx\") pod \"819ab857-6e46-4438-954a-d63331d501c3\" (UID: \"819ab857-6e46-4438-954a-d63331d501c3\") " Jul 06 18:05:14.007535 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.007508 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/819ab857-6e46-4438-954a-d63331d501c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "819ab857-6e46-4438-954a-d63331d501c3" (UID: "819ab857-6e46-4438-954a-d63331d501c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:05:14.009398 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.009368 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819ab857-6e46-4438-954a-d63331d501c3-kube-api-access-k8bdx" (OuterVolumeSpecName: "kube-api-access-k8bdx") pod "819ab857-6e46-4438-954a-d63331d501c3" (UID: "819ab857-6e46-4438-954a-d63331d501c3"). InnerVolumeSpecName "kube-api-access-k8bdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:05:14.009523 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.009477 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "819ab857-6e46-4438-954a-d63331d501c3" (UID: "819ab857-6e46-4438-954a-d63331d501c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:05:14.108576 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.108530 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/819ab857-6e46-4438-954a-d63331d501c3-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:14.108576 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.108578 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/819ab857-6e46-4438-954a-d63331d501c3-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:14.108788 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.108591 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8bdx\" (UniqueName: \"kubernetes.io/projected/819ab857-6e46-4438-954a-d63331d501c3-kube-api-access-k8bdx\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:14.417193 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.417158 2580 generic.go:356] "Generic (PLEG): container finished" podID="819ab857-6e46-4438-954a-d63331d501c3" containerID="b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857" exitCode=0 Jul 06 18:05:14.417408 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.417219 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" Jul 06 18:05:14.417408 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.417250 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" event={"ID":"819ab857-6e46-4438-954a-d63331d501c3","Type":"ContainerDied","Data":"b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857"} Jul 06 18:05:14.417408 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.417338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw" event={"ID":"819ab857-6e46-4438-954a-d63331d501c3","Type":"ContainerDied","Data":"126f47c7c2a06a28da9bf3f2b09d0ccd7d219436bfeb2b06c2f903ba35c3a3a3"} Jul 06 18:05:14.417408 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.417370 2580 scope.go:117] "RemoveContainer" containerID="b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857" Jul 06 18:05:14.426672 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.426652 2580 scope.go:117] "RemoveContainer" containerID="b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857" Jul 06 18:05:14.426956 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:05:14.426936 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857\": container with ID starting with b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857 not found: ID does not exist" containerID="b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857" Jul 06 18:05:14.427008 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.426964 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857"} err="failed to get container status \"b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857\": rpc error: code = NotFound desc = could not find container \"b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857\": container with ID starting with b620232f901e26223aa3cf5698ceef7093a62b041b0bd3a2418a14f59cd69857 not found: ID does not exist" Jul 06 18:05:14.438979 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.438943 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw"] Jul 06 18:05:14.444701 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.444672 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-2d5b0cda-f6db6cfdc-wjjsw"] Jul 06 18:05:14.772588 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:14.772553 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="819ab857-6e46-4438-954a-d63331d501c3" path="/var/lib/kubelet/pods/819ab857-6e46-4438-954a-d63331d501c3/volumes" Jul 06 18:05:20.501460 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.501413 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g"] Jul 06 18:05:20.501971 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.501955 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="819ab857-6e46-4438-954a-d63331d501c3" containerName="maas-api" Jul 06 18:05:20.501971 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.501973 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="819ab857-6e46-4438-954a-d63331d501c3" containerName="maas-api" Jul 06 18:05:20.502096 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.502080 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="819ab857-6e46-4438-954a-d63331d501c3" containerName="maas-api" Jul 06 18:05:20.506856 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.506831 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.509644 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.509615 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-ad92259c-serving-cert\"" Jul 06 18:05:20.516722 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.516393 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g"] Jul 06 18:05:20.669228 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.669181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgpnd\" (UniqueName: \"kubernetes.io/projected/538e2d07-c58b-441e-b155-abf0bf7b55e7-kube-api-access-pgpnd\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.669426 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.669319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538e2d07-c58b-441e-b155-abf0bf7b55e7-openshift-service-ca\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.669426 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.669392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538e2d07-c58b-441e-b155-abf0bf7b55e7-maas-api-tls\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.770645 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.770543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538e2d07-c58b-441e-b155-abf0bf7b55e7-openshift-service-ca\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.770923 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.770893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538e2d07-c58b-441e-b155-abf0bf7b55e7-maas-api-tls\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.771043 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.770975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgpnd\" (UniqueName: \"kubernetes.io/projected/538e2d07-c58b-441e-b155-abf0bf7b55e7-kube-api-access-pgpnd\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.771641 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.771610 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538e2d07-c58b-441e-b155-abf0bf7b55e7-openshift-service-ca\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.774983 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.774951 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538e2d07-c58b-441e-b155-abf0bf7b55e7-maas-api-tls\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.779033 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.779010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgpnd\" (UniqueName: \"kubernetes.io/projected/538e2d07-c58b-441e-b155-abf0bf7b55e7-kube-api-access-pgpnd\") pod \"maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.821090 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.821051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:20.972356 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:20.972322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g"] Jul 06 18:05:20.974278 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:05:20.974250 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538e2d07_c58b_441e_b155_abf0bf7b55e7.slice/crio-a74fd2f6640d41279f82a8ac22919ecbdc453ef2b36826d4a2a3faf5f19d4c58 WatchSource:0}: Error finding container a74fd2f6640d41279f82a8ac22919ecbdc453ef2b36826d4a2a3faf5f19d4c58: Status 404 returned error can't find the container with id a74fd2f6640d41279f82a8ac22919ecbdc453ef2b36826d4a2a3faf5f19d4c58 Jul 06 18:05:21.457170 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:21.457138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" event={"ID":"538e2d07-c58b-441e-b155-abf0bf7b55e7","Type":"ContainerStarted","Data":"f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b"} Jul 06 18:05:21.457354 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:21.457190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" event={"ID":"538e2d07-c58b-441e-b155-abf0bf7b55e7","Type":"ContainerStarted","Data":"a74fd2f6640d41279f82a8ac22919ecbdc453ef2b36826d4a2a3faf5f19d4c58"} Jul 06 18:05:21.457354 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:21.457218 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:21.476427 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:21.476368 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" podStartSLOduration=1.073374891 podStartE2EDuration="1.47634833s" totalImagesPullingTime="402.973439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:05:21.473263411 +0000 UTC m=+1615.223631013" watchObservedRunningTime="2026-07-06 18:05:21.47634833 +0000 UTC m=+1615.226715906" Jul 06 18:05:25.872822 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:25.872786 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g"] Jul 06 18:05:25.873331 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:25.872979 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" podUID="538e2d07-c58b-441e-b155-abf0bf7b55e7" containerName="maas-api" containerID="cri-o://f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b" gracePeriod=30 Jul 06 18:05:26.135553 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.135487 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:26.224606 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.224573 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538e2d07-c58b-441e-b155-abf0bf7b55e7-openshift-service-ca\") pod \"538e2d07-c58b-441e-b155-abf0bf7b55e7\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " Jul 06 18:05:26.224785 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.224619 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538e2d07-c58b-441e-b155-abf0bf7b55e7-maas-api-tls\") pod \"538e2d07-c58b-441e-b155-abf0bf7b55e7\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " Jul 06 18:05:26.224785 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.224665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgpnd\" (UniqueName: \"kubernetes.io/projected/538e2d07-c58b-441e-b155-abf0bf7b55e7-kube-api-access-pgpnd\") pod \"538e2d07-c58b-441e-b155-abf0bf7b55e7\" (UID: \"538e2d07-c58b-441e-b155-abf0bf7b55e7\") " Jul 06 18:05:26.225030 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.225010 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/538e2d07-c58b-441e-b155-abf0bf7b55e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "538e2d07-c58b-441e-b155-abf0bf7b55e7" (UID: "538e2d07-c58b-441e-b155-abf0bf7b55e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:05:26.226910 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.226875 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/538e2d07-c58b-441e-b155-abf0bf7b55e7-kube-api-access-pgpnd" (OuterVolumeSpecName: "kube-api-access-pgpnd") pod "538e2d07-c58b-441e-b155-abf0bf7b55e7" (UID: "538e2d07-c58b-441e-b155-abf0bf7b55e7"). InnerVolumeSpecName "kube-api-access-pgpnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:05:26.227013 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.226884 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/538e2d07-c58b-441e-b155-abf0bf7b55e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "538e2d07-c58b-441e-b155-abf0bf7b55e7" (UID: "538e2d07-c58b-441e-b155-abf0bf7b55e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:05:26.326152 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.326106 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/538e2d07-c58b-441e-b155-abf0bf7b55e7-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:26.326152 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.326143 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/538e2d07-c58b-441e-b155-abf0bf7b55e7-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:26.326152 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.326157 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgpnd\" (UniqueName: \"kubernetes.io/projected/538e2d07-c58b-441e-b155-abf0bf7b55e7-kube-api-access-pgpnd\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:26.488973 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.488932 2580 generic.go:356] "Generic (PLEG): container finished" podID="538e2d07-c58b-441e-b155-abf0bf7b55e7" containerID="f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b" exitCode=0 Jul 06 18:05:26.489142 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.488982 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" Jul 06 18:05:26.489142 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.489007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" event={"ID":"538e2d07-c58b-441e-b155-abf0bf7b55e7","Type":"ContainerDied","Data":"f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b"} Jul 06 18:05:26.489142 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.489084 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g" event={"ID":"538e2d07-c58b-441e-b155-abf0bf7b55e7","Type":"ContainerDied","Data":"a74fd2f6640d41279f82a8ac22919ecbdc453ef2b36826d4a2a3faf5f19d4c58"} Jul 06 18:05:26.489142 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.489107 2580 scope.go:117] "RemoveContainer" containerID="f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b" Jul 06 18:05:26.498987 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.498970 2580 scope.go:117] "RemoveContainer" containerID="f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b" Jul 06 18:05:26.499300 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:05:26.499266 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b\": container with ID starting with f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b not found: ID does not exist" containerID="f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b" Jul 06 18:05:26.499379 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.499321 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b"} err="failed to get container status \"f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b\": rpc error: code = NotFound desc = could not find container \"f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b\": container with ID starting with f854600393c0418c860999c16718aa7a85c724f5b5c9d41bc32a2491045e6b9b not found: ID does not exist" Jul 06 18:05:26.512203 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.512174 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g"] Jul 06 18:05:26.513736 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.513714 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-ad92259c-59df64cdb7-m9d4g"] Jul 06 18:05:26.768320 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:26.768233 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="538e2d07-c58b-441e-b155-abf0bf7b55e7" path="/var/lib/kubelet/pods/538e2d07-c58b-441e-b155-abf0bf7b55e7/volumes" Jul 06 18:05:38.905115 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.905083 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597"] Jul 06 18:05:38.905567 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.905551 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="538e2d07-c58b-441e-b155-abf0bf7b55e7" containerName="maas-api" Jul 06 18:05:38.905567 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.905567 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="538e2d07-c58b-441e-b155-abf0bf7b55e7" containerName="maas-api" Jul 06 18:05:38.905707 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.905693 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="538e2d07-c58b-441e-b155-abf0bf7b55e7" containerName="maas-api" Jul 06 18:05:38.910709 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.910686 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:38.913523 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.913502 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-9e236323-serving-cert\"" Jul 06 18:05:38.918456 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.918432 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597"] Jul 06 18:05:38.934640 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.934608 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:38.934842 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.934701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-openshift-service-ca\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:38.934842 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:38.934722 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nz2d\" (UniqueName: \"kubernetes.io/projected/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-kube-api-access-4nz2d\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.035771 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.035729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.035988 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.035874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-openshift-service-ca\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.035988 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.035904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nz2d\" (UniqueName: \"kubernetes.io/projected/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-kube-api-access-4nz2d\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.035988 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:05:39.035951 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-9e236323-serving-cert: secret "maas-api-e2e-derive-9e236323-serving-cert" not found Jul 06 18:05:39.036161 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:05:39.036027 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls podName:ec55db00-6f83-4f9a-8437-5ffef91a8fc3 nodeName:}" failed. No retries permitted until 2026-07-06 18:05:39.536005523 +0000 UTC m=+1633.286373101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls") pod "maas-api-e2e-derive-9e236323-78777bc74b-z2597" (UID: "ec55db00-6f83-4f9a-8437-5ffef91a8fc3") : secret "maas-api-e2e-derive-9e236323-serving-cert" not found Jul 06 18:05:39.036845 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.036820 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-openshift-service-ca\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.046518 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.046486 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nz2d\" (UniqueName: \"kubernetes.io/projected/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-kube-api-access-4nz2d\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.542006 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.541968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.544533 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.544509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls\") pod \"maas-api-e2e-derive-9e236323-78777bc74b-z2597\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.824966 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.824868 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:39.971447 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:39.971416 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597"] Jul 06 18:05:39.972844 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:05:39.972813 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec55db00_6f83_4f9a_8437_5ffef91a8fc3.slice/crio-4e19d2522b4f83484fd1eb279100b7f33caf66ce63990a3aca1cfa5951b1b118 WatchSource:0}: Error finding container 4e19d2522b4f83484fd1eb279100b7f33caf66ce63990a3aca1cfa5951b1b118: Status 404 returned error can't find the container with id 4e19d2522b4f83484fd1eb279100b7f33caf66ce63990a3aca1cfa5951b1b118 Jul 06 18:05:40.557904 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:40.557864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" event={"ID":"ec55db00-6f83-4f9a-8437-5ffef91a8fc3","Type":"ContainerStarted","Data":"6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a"} Jul 06 18:05:40.558078 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:40.557929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" event={"ID":"ec55db00-6f83-4f9a-8437-5ffef91a8fc3","Type":"ContainerStarted","Data":"4e19d2522b4f83484fd1eb279100b7f33caf66ce63990a3aca1cfa5951b1b118"} Jul 06 18:05:40.558078 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:40.557976 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:40.574780 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:40.574722 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" podStartSLOduration=2.138436158 podStartE2EDuration="2.574704938s" totalImagesPullingTime="436.26878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:05:40.573546006 +0000 UTC m=+1634.323913584" watchObservedRunningTime="2026-07-06 18:05:40.574704938 +0000 UTC m=+1634.325072513" Jul 06 18:05:43.777313 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:43.775382 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597"] Jul 06 18:05:43.777313 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:43.775729 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" podUID="ec55db00-6f83-4f9a-8437-5ffef91a8fc3" containerName="maas-api" containerID="cri-o://6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a" gracePeriod=30 Jul 06 18:05:44.028827 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.028762 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:44.086200 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.086167 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nz2d\" (UniqueName: \"kubernetes.io/projected/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-kube-api-access-4nz2d\") pod \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " Jul 06 18:05:44.086394 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.086246 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls\") pod \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " Jul 06 18:05:44.086394 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.086326 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-openshift-service-ca\") pod \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\" (UID: \"ec55db00-6f83-4f9a-8437-5ffef91a8fc3\") " Jul 06 18:05:44.086700 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.086665 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec55db00-6f83-4f9a-8437-5ffef91a8fc3" (UID: "ec55db00-6f83-4f9a-8437-5ffef91a8fc3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:05:44.088361 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.088329 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-kube-api-access-4nz2d" (OuterVolumeSpecName: "kube-api-access-4nz2d") pod "ec55db00-6f83-4f9a-8437-5ffef91a8fc3" (UID: "ec55db00-6f83-4f9a-8437-5ffef91a8fc3"). InnerVolumeSpecName "kube-api-access-4nz2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:05:44.088449 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.088333 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec55db00-6f83-4f9a-8437-5ffef91a8fc3" (UID: "ec55db00-6f83-4f9a-8437-5ffef91a8fc3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:05:44.187342 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.187272 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nz2d\" (UniqueName: \"kubernetes.io/projected/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-kube-api-access-4nz2d\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:44.187342 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.187337 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:44.187342 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.187348 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec55db00-6f83-4f9a-8437-5ffef91a8fc3-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:05:44.578444 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.578409 2580 generic.go:356] "Generic (PLEG): container finished" podID="ec55db00-6f83-4f9a-8437-5ffef91a8fc3" containerID="6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a" exitCode=0 Jul 06 18:05:44.578602 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.578453 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" Jul 06 18:05:44.578602 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.578474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" event={"ID":"ec55db00-6f83-4f9a-8437-5ffef91a8fc3","Type":"ContainerDied","Data":"6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a"} Jul 06 18:05:44.578602 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.578514 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597" event={"ID":"ec55db00-6f83-4f9a-8437-5ffef91a8fc3","Type":"ContainerDied","Data":"4e19d2522b4f83484fd1eb279100b7f33caf66ce63990a3aca1cfa5951b1b118"} Jul 06 18:05:44.578602 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.578536 2580 scope.go:117] "RemoveContainer" containerID="6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a" Jul 06 18:05:44.587191 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.587167 2580 scope.go:117] "RemoveContainer" containerID="6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a" Jul 06 18:05:44.587477 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:05:44.587457 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a\": container with ID starting with 6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a not found: ID does not exist" containerID="6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a" Jul 06 18:05:44.587539 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.587486 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a"} err="failed to get container status \"6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a\": rpc error: code = NotFound desc = could not find container \"6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a\": container with ID starting with 6d15dbf527650d43a6d8dfa997b75b07e06fb617ee438e04596cfcaa669fbb7a not found: ID does not exist" Jul 06 18:05:44.601392 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.601364 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597"] Jul 06 18:05:44.604816 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.604794 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-9e236323-78777bc74b-z2597"] Jul 06 18:05:44.768775 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:05:44.768743 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec55db00-6f83-4f9a-8437-5ffef91a8fc3" path="/var/lib/kubelet/pods/ec55db00-6f83-4f9a-8437-5ffef91a8fc3/volumes" Jul 06 18:07:27.527364 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.527323 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d"] Jul 06 18:07:27.527911 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.527715 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec55db00-6f83-4f9a-8437-5ffef91a8fc3" containerName="maas-api" Jul 06 18:07:27.527911 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.527730 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec55db00-6f83-4f9a-8437-5ffef91a8fc3" containerName="maas-api" Jul 06 18:07:27.527911 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.527831 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec55db00-6f83-4f9a-8437-5ffef91a8fc3" containerName="maas-api" Jul 06 18:07:27.530915 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.530894 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.534280 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.534256 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-2fc77276-serving-cert\"" Jul 06 18:07:27.649437 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.649398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkxzw\" (UniqueName: \"kubernetes.io/projected/21b28358-9b14-4e0d-bb93-293efd817068-kube-api-access-gkxzw\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.649618 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.649463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.649618 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.649513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b28358-9b14-4e0d-bb93-293efd817068-openshift-service-ca\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.750490 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.750448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.750490 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.750486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b28358-9b14-4e0d-bb93-293efd817068-openshift-service-ca\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.750764 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.750562 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkxzw\" (UniqueName: \"kubernetes.io/projected/21b28358-9b14-4e0d-bb93-293efd817068-kube-api-access-gkxzw\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.750764 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:07:27.750618 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-2fc77276-serving-cert: secret "maas-api-e2e-mt-2fc77276-serving-cert" not found Jul 06 18:07:27.750764 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:07:27.750714 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls podName:21b28358-9b14-4e0d-bb93-293efd817068 nodeName:}" failed. No retries permitted until 2026-07-06 18:07:28.250691132 +0000 UTC m=+1742.001058685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls") pod "maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" (UID: "21b28358-9b14-4e0d-bb93-293efd817068") : secret "maas-api-e2e-mt-2fc77276-serving-cert" not found Jul 06 18:07:27.751254 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.751232 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b28358-9b14-4e0d-bb93-293efd817068-openshift-service-ca\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:27.767377 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:27.767347 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkxzw\" (UniqueName: \"kubernetes.io/projected/21b28358-9b14-4e0d-bb93-293efd817068-kube-api-access-gkxzw\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:28.255727 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:28.255678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:28.258058 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:28.258029 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls\") pod \"maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:28.443892 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:28.443854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:49.498599 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:49.498557 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d"] Jul 06 18:07:58.014374 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:58.014339 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d"] Jul 06 18:07:58.135716 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:58.135677 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d"] Jul 06 18:07:58.137006 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:07:58.136976 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b28358_9b14_4e0d_bb93_293efd817068.slice/crio-411229edd8c33d0bc934d5e48433c70f876e3ceacfd4409e36ffc9fb3d1ebca7 WatchSource:0}: Error finding container 411229edd8c33d0bc934d5e48433c70f876e3ceacfd4409e36ffc9fb3d1ebca7: Status 404 returned error can't find the container with id 411229edd8c33d0bc934d5e48433c70f876e3ceacfd4409e36ffc9fb3d1ebca7 Jul 06 18:07:58.229406 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:58.229366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" event={"ID":"21b28358-9b14-4e0d-bb93-293efd817068","Type":"ContainerStarted","Data":"411229edd8c33d0bc934d5e48433c70f876e3ceacfd4409e36ffc9fb3d1ebca7"} Jul 06 18:07:59.236590 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.236550 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" event={"ID":"21b28358-9b14-4e0d-bb93-293efd817068","Type":"ContainerStarted","Data":"91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2"} Jul 06 18:07:59.237059 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.236606 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" podUID="21b28358-9b14-4e0d-bb93-293efd817068" containerName="maas-api" containerID="cri-o://91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2" gracePeriod=30 Jul 06 18:07:59.237059 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.236681 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:59.254456 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.254405 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" podStartSLOduration=31.767719296 podStartE2EDuration="32.254384502s" totalImagesPullingTime="486.665206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:07:59.253599724 +0000 UTC m=+1773.003967324" watchObservedRunningTime="2026-07-06 18:07:59.254384502 +0000 UTC m=+1773.004752080" Jul 06 18:07:59.486372 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.486347 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:07:59.556027 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.555939 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls\") pod \"21b28358-9b14-4e0d-bb93-293efd817068\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " Jul 06 18:07:59.556027 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.556001 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkxzw\" (UniqueName: \"kubernetes.io/projected/21b28358-9b14-4e0d-bb93-293efd817068-kube-api-access-gkxzw\") pod \"21b28358-9b14-4e0d-bb93-293efd817068\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " Jul 06 18:07:59.556259 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.556044 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b28358-9b14-4e0d-bb93-293efd817068-openshift-service-ca\") pod \"21b28358-9b14-4e0d-bb93-293efd817068\" (UID: \"21b28358-9b14-4e0d-bb93-293efd817068\") " Jul 06 18:07:59.556484 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.556458 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21b28358-9b14-4e0d-bb93-293efd817068-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21b28358-9b14-4e0d-bb93-293efd817068" (UID: "21b28358-9b14-4e0d-bb93-293efd817068"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:07:59.558115 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.558089 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21b28358-9b14-4e0d-bb93-293efd817068-kube-api-access-gkxzw" (OuterVolumeSpecName: "kube-api-access-gkxzw") pod "21b28358-9b14-4e0d-bb93-293efd817068" (UID: "21b28358-9b14-4e0d-bb93-293efd817068"). InnerVolumeSpecName "kube-api-access-gkxzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:07:59.558196 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.558133 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21b28358-9b14-4e0d-bb93-293efd817068" (UID: "21b28358-9b14-4e0d-bb93-293efd817068"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:07:59.656914 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.656870 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkxzw\" (UniqueName: \"kubernetes.io/projected/21b28358-9b14-4e0d-bb93-293efd817068-kube-api-access-gkxzw\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:07:59.656914 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.656905 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b28358-9b14-4e0d-bb93-293efd817068-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:07:59.656914 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:07:59.656917 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b28358-9b14-4e0d-bb93-293efd817068-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:08:00.242640 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.242605 2580 generic.go:356] "Generic (PLEG): container finished" podID="21b28358-9b14-4e0d-bb93-293efd817068" containerID="91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2" exitCode=0 Jul 06 18:08:00.242640 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.242646 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" Jul 06 18:08:00.243147 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.242668 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" event={"ID":"21b28358-9b14-4e0d-bb93-293efd817068","Type":"ContainerDied","Data":"91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2"} Jul 06 18:08:00.243147 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.242710 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d" event={"ID":"21b28358-9b14-4e0d-bb93-293efd817068","Type":"ContainerDied","Data":"411229edd8c33d0bc934d5e48433c70f876e3ceacfd4409e36ffc9fb3d1ebca7"} Jul 06 18:08:00.243147 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.242733 2580 scope.go:117] "RemoveContainer" containerID="91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2" Jul 06 18:08:00.252258 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.252238 2580 scope.go:117] "RemoveContainer" containerID="91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2" Jul 06 18:08:00.252572 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:08:00.252554 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2\": container with ID starting with 91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2 not found: ID does not exist" containerID="91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2" Jul 06 18:08:00.252651 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.252581 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2"} err="failed to get container status \"91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2\": rpc error: code = NotFound desc = could not find container \"91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2\": container with ID starting with 91596d2a68b8e791e25ff81a800b2718a5ed5118e49de5729c5bc0a06d25acc2 not found: ID does not exist" Jul 06 18:08:00.264385 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.264355 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d"] Jul 06 18:08:00.267903 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.267879 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-2fc77276-79dcd65f5c-l786d"] Jul 06 18:08:00.768517 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:00.768480 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21b28358-9b14-4e0d-bb93-293efd817068" path="/var/lib/kubelet/pods/21b28358-9b14-4e0d-bb93-293efd817068/volumes" Jul 06 18:08:18.892044 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:18.891955 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj"] Jul 06 18:08:18.892584 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:18.892565 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21b28358-9b14-4e0d-bb93-293efd817068" containerName="maas-api" Jul 06 18:08:18.892679 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:18.892587 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="21b28358-9b14-4e0d-bb93-293efd817068" containerName="maas-api" Jul 06 18:08:18.892739 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:18.892714 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="21b28358-9b14-4e0d-bb93-293efd817068" containerName="maas-api" Jul 06 18:08:18.897041 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:18.897021 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:18.905523 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:18.905495 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-214076ae-serving-cert\"" Jul 06 18:08:18.913929 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:18.913904 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj"] Jul 06 18:08:19.043240 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.043202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng6st\" (UniqueName: \"kubernetes.io/projected/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-kube-api-access-ng6st\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.043443 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.043262 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-openshift-service-ca\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.043443 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.043357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.144359 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.144250 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng6st\" (UniqueName: \"kubernetes.io/projected/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-kube-api-access-ng6st\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.144359 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.144329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-openshift-service-ca\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.144622 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.144385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.144622 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:08:19.144512 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-214076ae-serving-cert: secret "maas-api-e2e-mt-214076ae-serving-cert" not found Jul 06 18:08:19.144622 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:08:19.144591 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls podName:b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79 nodeName:}" failed. No retries permitted until 2026-07-06 18:08:19.644569986 +0000 UTC m=+1793.394937543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls") pod "maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" (UID: "b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79") : secret "maas-api-e2e-mt-214076ae-serving-cert" not found Jul 06 18:08:19.145041 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.145018 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-openshift-service-ca\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.168752 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.168721 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng6st\" (UniqueName: \"kubernetes.io/projected/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-kube-api-access-ng6st\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.649472 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.649426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.651885 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.651859 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls\") pod \"maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.808093 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.808049 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:19.952046 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:19.952009 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj"] Jul 06 18:08:19.954115 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:08:19.954088 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a626f3_ebda_4baa_a2dc_0aea7f9b8f79.slice/crio-24a213d86d5af3cdaf90d1ec8e427ad1ed3a86f064258dcd7b834f65966a157a WatchSource:0}: Error finding container 24a213d86d5af3cdaf90d1ec8e427ad1ed3a86f064258dcd7b834f65966a157a: Status 404 returned error can't find the container with id 24a213d86d5af3cdaf90d1ec8e427ad1ed3a86f064258dcd7b834f65966a157a Jul 06 18:08:20.347679 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:20.347636 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" event={"ID":"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79","Type":"ContainerStarted","Data":"24a213d86d5af3cdaf90d1ec8e427ad1ed3a86f064258dcd7b834f65966a157a"} Jul 06 18:08:21.357242 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:21.357191 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" event={"ID":"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79","Type":"ContainerStarted","Data":"48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5"} Jul 06 18:08:21.357687 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:21.357331 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:08:21.374310 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:21.374235 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" podStartSLOduration=2.4370958910000002 podStartE2EDuration="3.374219139s" totalImagesPullingTime="937.123248ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:08:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:08:21.373363584 +0000 UTC m=+1795.123731159" watchObservedRunningTime="2026-07-06 18:08:21.374219139 +0000 UTC m=+1795.124586716" Jul 06 18:08:26.787623 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:26.787591 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:08:26.790153 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:26.790127 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:08:27.368983 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:08:27.368955 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:09:01.169185 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.169138 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj"] Jul 06 18:09:01.169777 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.169428 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" podUID="b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" containerName="maas-api" containerID="cri-o://48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5" gracePeriod=30 Jul 06 18:09:01.421649 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.421576 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:09:01.546601 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.546567 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng6st\" (UniqueName: \"kubernetes.io/projected/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-kube-api-access-ng6st\") pod \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " Jul 06 18:09:01.546811 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.546672 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-openshift-service-ca\") pod \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " Jul 06 18:09:01.546811 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.546699 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls\") pod \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\" (UID: \"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79\") " Jul 06 18:09:01.547085 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.547053 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" (UID: "b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:09:01.548991 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.548948 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-kube-api-access-ng6st" (OuterVolumeSpecName: "kube-api-access-ng6st") pod "b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" (UID: "b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79"). InnerVolumeSpecName "kube-api-access-ng6st". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:09:01.548991 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.548979 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" (UID: "b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:09:01.556213 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.556180 2580 generic.go:356] "Generic (PLEG): container finished" podID="b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" containerID="48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5" exitCode=0 Jul 06 18:09:01.556361 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.556236 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" Jul 06 18:09:01.556361 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.556313 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" event={"ID":"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79","Type":"ContainerDied","Data":"48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5"} Jul 06 18:09:01.556451 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.556371 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj" event={"ID":"b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79","Type":"ContainerDied","Data":"24a213d86d5af3cdaf90d1ec8e427ad1ed3a86f064258dcd7b834f65966a157a"} Jul 06 18:09:01.556451 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.556403 2580 scope.go:117] "RemoveContainer" containerID="48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5" Jul 06 18:09:01.572419 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.572399 2580 scope.go:117] "RemoveContainer" containerID="48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5" Jul 06 18:09:01.572727 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:09:01.572707 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5\": container with ID starting with 48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5 not found: ID does not exist" containerID="48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5" Jul 06 18:09:01.572768 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.572744 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5"} err="failed to get container status \"48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5\": rpc error: code = NotFound desc = could not find container \"48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5\": container with ID starting with 48115b7e375e757ce1a0b0d22a3cab967259cfecf5660dc374ca7ae1a02351b5 not found: ID does not exist" Jul 06 18:09:01.585223 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.585189 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj"] Jul 06 18:09:01.588573 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.588548 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-214076ae-65c755d7f7-8cgxj"] Jul 06 18:09:01.648378 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.648340 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:09:01.648378 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.648375 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:09:01.648586 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:01.648391 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng6st\" (UniqueName: \"kubernetes.io/projected/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79-kube-api-access-ng6st\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:09:02.768110 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:09:02.768071 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" path="/var/lib/kubelet/pods/b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79/volumes" Jul 06 18:10:55.633135 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.633095 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r"] Jul 06 18:10:55.635734 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.633707 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" containerName="maas-api" Jul 06 18:10:55.635734 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.633727 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" containerName="maas-api" Jul 06 18:10:55.635734 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.633829 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8a626f3-ebda-4baa-a2dc-0aea7f9b8f79" containerName="maas-api" Jul 06 18:10:55.637259 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.637238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.640335 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.640284 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-b635d0-serving-cert\"" Jul 06 18:10:55.648060 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.648034 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r"] Jul 06 18:10:55.699920 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.699886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f74bc46-74ee-467c-96ab-d389991963e1-maas-api-tls\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.699920 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.699923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f74bc46-74ee-467c-96ab-d389991963e1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.700163 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.700018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqb6\" (UniqueName: \"kubernetes.io/projected/9f74bc46-74ee-467c-96ab-d389991963e1-kube-api-access-pgqb6\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.800964 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.800926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqb6\" (UniqueName: \"kubernetes.io/projected/9f74bc46-74ee-467c-96ab-d389991963e1-kube-api-access-pgqb6\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.801175 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.801099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f74bc46-74ee-467c-96ab-d389991963e1-maas-api-tls\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.801175 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.801121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f74bc46-74ee-467c-96ab-d389991963e1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.801754 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.801736 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f74bc46-74ee-467c-96ab-d389991963e1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.803616 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.803594 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f74bc46-74ee-467c-96ab-d389991963e1-maas-api-tls\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.809615 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.809589 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqb6\" (UniqueName: \"kubernetes.io/projected/9f74bc46-74ee-467c-96ab-d389991963e1-kube-api-access-pgqb6\") pod \"maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:55.951991 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:55.951894 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:56.110634 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:56.110607 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r"] Jul 06 18:10:56.112276 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:10:56.112244 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f74bc46_74ee_467c_96ab_d389991963e1.slice/crio-7230ed6337c03631efd206d019b8eb52361a8567abb3579da21f103fa06a33d9 WatchSource:0}: Error finding container 7230ed6337c03631efd206d019b8eb52361a8567abb3579da21f103fa06a33d9: Status 404 returned error can't find the container with id 7230ed6337c03631efd206d019b8eb52361a8567abb3579da21f103fa06a33d9 Jul 06 18:10:56.113555 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:56.113537 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:10:57.109013 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:57.108982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" event={"ID":"9f74bc46-74ee-467c-96ab-d389991963e1","Type":"ContainerStarted","Data":"b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629"} Jul 06 18:10:57.109460 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:57.109037 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" event={"ID":"9f74bc46-74ee-467c-96ab-d389991963e1","Type":"ContainerStarted","Data":"7230ed6337c03631efd206d019b8eb52361a8567abb3579da21f103fa06a33d9"} Jul 06 18:10:57.109460 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:57.109064 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:10:57.132702 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:10:57.132649 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" podStartSLOduration=1.656325059 podStartE2EDuration="2.132632828s" totalImagesPullingTime="476.307769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:10:57.129879105 +0000 UTC m=+1950.880246692" watchObservedRunningTime="2026-07-06 18:10:57.132632828 +0000 UTC m=+1950.883000403" Jul 06 18:11:03.115789 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.115735 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.44:8443/health\": dial tcp 10.132.0.44:8443: connect: connection refused" Jul 06 18:11:03.646516 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.646463 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4"] Jul 06 18:11:03.650652 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.650623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.653775 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.653749 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-3168b2-serving-cert\"" Jul 06 18:11:03.663752 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.663722 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4"] Jul 06 18:11:03.781060 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.781023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4sk8\" (UniqueName: \"kubernetes.io/projected/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-kube-api-access-t4sk8\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.781250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.781078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.781250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.781172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.882060 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.882014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4sk8\" (UniqueName: \"kubernetes.io/projected/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-kube-api-access-t4sk8\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.882246 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.882078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.882246 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.882119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.882374 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:11:03.882279 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-3168b2-serving-cert: secret "maas-api-e2e-inf-b-3168b2-serving-cert" not found Jul 06 18:11:03.882447 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:11:03.882382 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls podName:2efa3b37-9215-4ea1-b3ba-05a81289a0f1 nodeName:}" failed. No retries permitted until 2026-07-06 18:11:04.382357558 +0000 UTC m=+1958.132725111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls") pod "maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" (UID: "2efa3b37-9215-4ea1-b3ba-05a81289a0f1") : secret "maas-api-e2e-inf-b-3168b2-serving-cert" not found Jul 06 18:11:03.882865 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.882844 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:03.898246 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:03.898174 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4sk8\" (UniqueName: \"kubernetes.io/projected/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-kube-api-access-t4sk8\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:04.387595 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:04.387553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:04.390141 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:04.390114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls\") pod \"maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:04.565727 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:04.565683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:04.708818 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:04.708788 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4"] Jul 06 18:11:04.714063 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:11:04.714031 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efa3b37_9215_4ea1_b3ba_05a81289a0f1.slice/crio-3b583b87995e10a481cf60a189872018f0268b4eadc93557aeaba09bc832ee61 WatchSource:0}: Error finding container 3b583b87995e10a481cf60a189872018f0268b4eadc93557aeaba09bc832ee61: Status 404 returned error can't find the container with id 3b583b87995e10a481cf60a189872018f0268b4eadc93557aeaba09bc832ee61 Jul 06 18:11:05.151452 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:05.151420 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" event={"ID":"2efa3b37-9215-4ea1-b3ba-05a81289a0f1","Type":"ContainerStarted","Data":"7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38"} Jul 06 18:11:05.151617 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:05.151474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" event={"ID":"2efa3b37-9215-4ea1-b3ba-05a81289a0f1","Type":"ContainerStarted","Data":"3b583b87995e10a481cf60a189872018f0268b4eadc93557aeaba09bc832ee61"} Jul 06 18:11:05.151617 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:05.151531 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:05.169550 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:05.169493 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" podStartSLOduration=1.8313316099999999 podStartE2EDuration="2.16947354s" totalImagesPullingTime="338.14193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:11:05.167708758 +0000 UTC m=+1958.918076349" watchObservedRunningTime="2026-07-06 18:11:05.16947354 +0000 UTC m=+1958.919841116" Jul 06 18:11:08.114876 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:08.114828 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.44:8443/health\": dial tcp 10.132.0.44:8443: connect: connection refused" Jul 06 18:11:11.158378 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:11.158331 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" podUID="2efa3b37-9215-4ea1-b3ba-05a81289a0f1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.45:8443/health\": dial tcp 10.132.0.45:8443: connect: connection refused" Jul 06 18:11:13.119139 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:13.119111 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:11:16.162384 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:16.162347 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:11:58.109979 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.109943 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r"] Jul 06 18:11:58.110642 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.110177 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" containerName="maas-api" containerID="cri-o://b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629" gracePeriod=30 Jul 06 18:11:58.114271 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.114238 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.44:8443/health\": dial tcp 10.132.0.44:8443: connect: connection refused" Jul 06 18:11:58.360981 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.360909 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:11:58.412848 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.412819 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f74bc46-74ee-467c-96ab-d389991963e1-openshift-service-ca\") pod \"9f74bc46-74ee-467c-96ab-d389991963e1\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " Jul 06 18:11:58.412848 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.412852 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgqb6\" (UniqueName: \"kubernetes.io/projected/9f74bc46-74ee-467c-96ab-d389991963e1-kube-api-access-pgqb6\") pod \"9f74bc46-74ee-467c-96ab-d389991963e1\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " Jul 06 18:11:58.413162 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.412922 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f74bc46-74ee-467c-96ab-d389991963e1-maas-api-tls\") pod \"9f74bc46-74ee-467c-96ab-d389991963e1\" (UID: \"9f74bc46-74ee-467c-96ab-d389991963e1\") " Jul 06 18:11:58.413238 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.413205 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f74bc46-74ee-467c-96ab-d389991963e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f74bc46-74ee-467c-96ab-d389991963e1" (UID: "9f74bc46-74ee-467c-96ab-d389991963e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:11:58.415116 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.415087 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f74bc46-74ee-467c-96ab-d389991963e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f74bc46-74ee-467c-96ab-d389991963e1" (UID: "9f74bc46-74ee-467c-96ab-d389991963e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:11:58.415366 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.415340 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f74bc46-74ee-467c-96ab-d389991963e1-kube-api-access-pgqb6" (OuterVolumeSpecName: "kube-api-access-pgqb6") pod "9f74bc46-74ee-467c-96ab-d389991963e1" (UID: "9f74bc46-74ee-467c-96ab-d389991963e1"). InnerVolumeSpecName "kube-api-access-pgqb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:11:58.430542 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.430501 2580 generic.go:356] "Generic (PLEG): container finished" podID="9f74bc46-74ee-467c-96ab-d389991963e1" containerID="b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629" exitCode=0 Jul 06 18:11:58.430719 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.430608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" event={"ID":"9f74bc46-74ee-467c-96ab-d389991963e1","Type":"ContainerDied","Data":"b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629"} Jul 06 18:11:58.430719 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.430693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" event={"ID":"9f74bc46-74ee-467c-96ab-d389991963e1","Type":"ContainerDied","Data":"7230ed6337c03631efd206d019b8eb52361a8567abb3579da21f103fa06a33d9"} Jul 06 18:11:58.430855 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.430723 2580 scope.go:117] "RemoveContainer" containerID="b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629" Jul 06 18:11:58.430855 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.430732 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r" Jul 06 18:11:58.442066 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.442048 2580 scope.go:117] "RemoveContainer" containerID="b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629" Jul 06 18:11:58.442454 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:11:58.442435 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629\": container with ID starting with b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629 not found: ID does not exist" containerID="b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629" Jul 06 18:11:58.442534 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.442463 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629"} err="failed to get container status \"b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629\": rpc error: code = NotFound desc = could not find container \"b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629\": container with ID starting with b6ae0f36787991c93379b98ebad13bc9a2ded9961cbb9e1bf7a3201fdd27e629 not found: ID does not exist" Jul 06 18:11:58.453846 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.453811 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r"] Jul 06 18:11:58.458958 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.458929 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b635d0-5b74c5b97-9q48r"] Jul 06 18:11:58.514093 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.514058 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f74bc46-74ee-467c-96ab-d389991963e1-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:11:58.514093 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.514090 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f74bc46-74ee-467c-96ab-d389991963e1-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:11:58.514093 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.514099 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgqb6\" (UniqueName: \"kubernetes.io/projected/9f74bc46-74ee-467c-96ab-d389991963e1-kube-api-access-pgqb6\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:11:58.768867 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:11:58.768830 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" path="/var/lib/kubelet/pods/9f74bc46-74ee-467c-96ab-d389991963e1/volumes" Jul 06 18:12:41.261454 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.261414 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4"] Jul 06 18:12:41.262040 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.261697 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" podUID="2efa3b37-9215-4ea1-b3ba-05a81289a0f1" containerName="maas-api" containerID="cri-o://7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38" gracePeriod=30 Jul 06 18:12:41.518531 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.518470 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:12:41.615697 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.615658 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4sk8\" (UniqueName: \"kubernetes.io/projected/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-kube-api-access-t4sk8\") pod \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " Jul 06 18:12:41.615921 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.615787 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-openshift-service-ca\") pod \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " Jul 06 18:12:41.615921 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.615816 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls\") pod \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\" (UID: \"2efa3b37-9215-4ea1-b3ba-05a81289a0f1\") " Jul 06 18:12:41.616214 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.616188 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2efa3b37-9215-4ea1-b3ba-05a81289a0f1" (UID: "2efa3b37-9215-4ea1-b3ba-05a81289a0f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:12:41.617827 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.617792 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-kube-api-access-t4sk8" (OuterVolumeSpecName: "kube-api-access-t4sk8") pod "2efa3b37-9215-4ea1-b3ba-05a81289a0f1" (UID: "2efa3b37-9215-4ea1-b3ba-05a81289a0f1"). InnerVolumeSpecName "kube-api-access-t4sk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:12:41.617940 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.617847 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2efa3b37-9215-4ea1-b3ba-05a81289a0f1" (UID: "2efa3b37-9215-4ea1-b3ba-05a81289a0f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:12:41.643636 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.643602 2580 generic.go:356] "Generic (PLEG): container finished" podID="2efa3b37-9215-4ea1-b3ba-05a81289a0f1" containerID="7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38" exitCode=0 Jul 06 18:12:41.643848 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.643645 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" Jul 06 18:12:41.643848 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.643686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" event={"ID":"2efa3b37-9215-4ea1-b3ba-05a81289a0f1","Type":"ContainerDied","Data":"7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38"} Jul 06 18:12:41.643848 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.643742 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4" event={"ID":"2efa3b37-9215-4ea1-b3ba-05a81289a0f1","Type":"ContainerDied","Data":"3b583b87995e10a481cf60a189872018f0268b4eadc93557aeaba09bc832ee61"} Jul 06 18:12:41.643848 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.643775 2580 scope.go:117] "RemoveContainer" containerID="7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38" Jul 06 18:12:41.656043 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.656024 2580 scope.go:117] "RemoveContainer" containerID="7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38" Jul 06 18:12:41.656459 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:12:41.656437 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38\": container with ID starting with 7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38 not found: ID does not exist" containerID="7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38" Jul 06 18:12:41.656555 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.656467 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38"} err="failed to get container status \"7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38\": rpc error: code = NotFound desc = could not find container \"7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38\": container with ID starting with 7110e0fdeb8b78309825bdf1385693a16965647884ea91221cf35ccf90255a38 not found: ID does not exist" Jul 06 18:12:41.665669 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.665633 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4"] Jul 06 18:12:41.668040 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.668014 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-3168b2-6f6d795674-ljzn4"] Jul 06 18:12:41.716697 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.716662 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4sk8\" (UniqueName: \"kubernetes.io/projected/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-kube-api-access-t4sk8\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:12:41.716697 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.716691 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:12:41.716697 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:41.716704 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2efa3b37-9215-4ea1-b3ba-05a81289a0f1-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:12:42.768196 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:42.768161 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2efa3b37-9215-4ea1-b3ba-05a81289a0f1" path="/var/lib/kubelet/pods/2efa3b37-9215-4ea1-b3ba-05a81289a0f1/volumes" Jul 06 18:12:51.233008 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.232959 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl"] Jul 06 18:12:51.233640 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.233603 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" containerName="maas-api" Jul 06 18:12:51.233640 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.233624 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" containerName="maas-api" Jul 06 18:12:51.233782 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.233662 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2efa3b37-9215-4ea1-b3ba-05a81289a0f1" containerName="maas-api" Jul 06 18:12:51.233782 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.233671 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2efa3b37-9215-4ea1-b3ba-05a81289a0f1" containerName="maas-api" Jul 06 18:12:51.233782 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.233758 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f74bc46-74ee-467c-96ab-d389991963e1" containerName="maas-api" Jul 06 18:12:51.233782 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.233775 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2efa3b37-9215-4ea1-b3ba-05a81289a0f1" containerName="maas-api" Jul 06 18:12:51.238621 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.238591 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.241701 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.241677 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-453d8d-serving-cert\"" Jul 06 18:12:51.251819 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.251790 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl"] Jul 06 18:12:51.306055 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.306023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-maas-api-tls\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.306232 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.306064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gppcq\" (UniqueName: \"kubernetes.io/projected/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-kube-api-access-gppcq\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.306232 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.306129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-openshift-service-ca\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.407403 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.407369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-maas-api-tls\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.407586 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.407419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gppcq\" (UniqueName: \"kubernetes.io/projected/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-kube-api-access-gppcq\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.407586 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.407480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-openshift-service-ca\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.408209 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.408187 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-openshift-service-ca\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.409877 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.409854 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-maas-api-tls\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.415654 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.415628 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gppcq\" (UniqueName: \"kubernetes.io/projected/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-kube-api-access-gppcq\") pod \"maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.551746 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.551651 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:51.694003 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:51.693969 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl"] Jul 06 18:12:51.697067 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:12:51.697038 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e62899_4e17_4b75_8e8e_9193abb6ba9b.slice/crio-3b540ef9feac37015071a0b6283d6f88b64e94eecced4940d20121a3c6aa657c WatchSource:0}: Error finding container 3b540ef9feac37015071a0b6283d6f88b64e94eecced4940d20121a3c6aa657c: Status 404 returned error can't find the container with id 3b540ef9feac37015071a0b6283d6f88b64e94eecced4940d20121a3c6aa657c Jul 06 18:12:52.698665 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:52.698629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" event={"ID":"c1e62899-4e17-4b75-8e8e-9193abb6ba9b","Type":"ContainerStarted","Data":"711784d3b07bba55bbca022e05c1716a173bef0b529d61206eff6a3728c6aba7"} Jul 06 18:12:52.699055 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:52.698682 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" event={"ID":"c1e62899-4e17-4b75-8e8e-9193abb6ba9b","Type":"ContainerStarted","Data":"3b540ef9feac37015071a0b6283d6f88b64e94eecced4940d20121a3c6aa657c"} Jul 06 18:12:52.699055 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:52.698712 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:52.718040 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:52.717992 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" podStartSLOduration=1.307106164 podStartE2EDuration="1.717974846s" totalImagesPullingTime="410.868682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:12:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:12:52.717171583 +0000 UTC m=+2066.467539159" watchObservedRunningTime="2026-07-06 18:12:52.717974846 +0000 UTC m=+2066.468342421" Jul 06 18:12:58.709479 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:58.709446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:12:59.492105 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.492061 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln"] Jul 06 18:12:59.496238 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.496216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.498974 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.498955 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-f63d8f-serving-cert\"" Jul 06 18:12:59.508895 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.508866 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln"] Jul 06 18:12:59.588433 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.588399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2h4m\" (UniqueName: \"kubernetes.io/projected/4897a06b-50a8-484c-9625-7983bba68c1f-kube-api-access-d2h4m\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.588616 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.588526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4897a06b-50a8-484c-9625-7983bba68c1f-openshift-service-ca\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.588666 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.588650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.689685 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.689640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.689883 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.689694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2h4m\" (UniqueName: \"kubernetes.io/projected/4897a06b-50a8-484c-9625-7983bba68c1f-kube-api-access-d2h4m\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.689883 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.689748 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4897a06b-50a8-484c-9625-7983bba68c1f-openshift-service-ca\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.689883 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:12:59.689812 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-f63d8f-serving-cert: secret "maas-api-e2e-api-b-f63d8f-serving-cert" not found Jul 06 18:12:59.690006 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:12:59.689894 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls podName:4897a06b-50a8-484c-9625-7983bba68c1f nodeName:}" failed. No retries permitted until 2026-07-06 18:13:00.189872265 +0000 UTC m=+2073.940239819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls") pod "maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" (UID: "4897a06b-50a8-484c-9625-7983bba68c1f") : secret "maas-api-e2e-api-b-f63d8f-serving-cert" not found Jul 06 18:12:59.690374 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.690358 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4897a06b-50a8-484c-9625-7983bba68c1f-openshift-service-ca\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:12:59.699135 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:12:59.699100 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2h4m\" (UniqueName: \"kubernetes.io/projected/4897a06b-50a8-484c-9625-7983bba68c1f-kube-api-access-d2h4m\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:00.193471 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:00.193435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:00.196049 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:00.196024 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls\") pod \"maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:00.409526 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:00.409488 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:00.560516 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:00.560473 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln"] Jul 06 18:13:00.562500 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:13:00.562472 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4897a06b_50a8_484c_9625_7983bba68c1f.slice/crio-f0e7062c4c67416ec0b766671e000f573554d91a1845c271bdccc20d5f7c0932 WatchSource:0}: Error finding container f0e7062c4c67416ec0b766671e000f573554d91a1845c271bdccc20d5f7c0932: Status 404 returned error can't find the container with id f0e7062c4c67416ec0b766671e000f573554d91a1845c271bdccc20d5f7c0932 Jul 06 18:13:00.740757 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:00.740725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" event={"ID":"4897a06b-50a8-484c-9625-7983bba68c1f","Type":"ContainerStarted","Data":"f0e7062c4c67416ec0b766671e000f573554d91a1845c271bdccc20d5f7c0932"} Jul 06 18:13:01.747511 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:01.747472 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" event={"ID":"4897a06b-50a8-484c-9625-7983bba68c1f","Type":"ContainerStarted","Data":"1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d"} Jul 06 18:13:01.747925 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:01.747544 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:01.766218 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:01.766166 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" podStartSLOduration=2.279723281 podStartE2EDuration="2.766149995s" totalImagesPullingTime="486.426714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:13:01.764035907 +0000 UTC m=+2075.514403484" watchObservedRunningTime="2026-07-06 18:13:01.766149995 +0000 UTC m=+2075.516517572" Jul 06 18:13:07.757898 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:07.757869 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:12.627733 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:12.627697 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl"] Jul 06 18:13:12.628260 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:12.628008 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" podUID="c1e62899-4e17-4b75-8e8e-9193abb6ba9b" containerName="maas-api" containerID="cri-o://711784d3b07bba55bbca022e05c1716a173bef0b529d61206eff6a3728c6aba7" gracePeriod=30 Jul 06 18:13:12.807769 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:12.807728 2580 generic.go:356] "Generic (PLEG): container finished" podID="c1e62899-4e17-4b75-8e8e-9193abb6ba9b" containerID="711784d3b07bba55bbca022e05c1716a173bef0b529d61206eff6a3728c6aba7" exitCode=0 Jul 06 18:13:12.807911 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:12.807802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" event={"ID":"c1e62899-4e17-4b75-8e8e-9193abb6ba9b","Type":"ContainerDied","Data":"711784d3b07bba55bbca022e05c1716a173bef0b529d61206eff6a3728c6aba7"} Jul 06 18:13:12.878531 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:12.878460 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:13:13.016384 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.016351 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gppcq\" (UniqueName: \"kubernetes.io/projected/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-kube-api-access-gppcq\") pod \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " Jul 06 18:13:13.016582 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.016473 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-maas-api-tls\") pod \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " Jul 06 18:13:13.016582 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.016502 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-openshift-service-ca\") pod \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\" (UID: \"c1e62899-4e17-4b75-8e8e-9193abb6ba9b\") " Jul 06 18:13:13.016934 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.016891 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1e62899-4e17-4b75-8e8e-9193abb6ba9b" (UID: "c1e62899-4e17-4b75-8e8e-9193abb6ba9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:13:13.018615 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.018587 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-kube-api-access-gppcq" (OuterVolumeSpecName: "kube-api-access-gppcq") pod "c1e62899-4e17-4b75-8e8e-9193abb6ba9b" (UID: "c1e62899-4e17-4b75-8e8e-9193abb6ba9b"). InnerVolumeSpecName "kube-api-access-gppcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:13:13.018703 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.018633 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1e62899-4e17-4b75-8e8e-9193abb6ba9b" (UID: "c1e62899-4e17-4b75-8e8e-9193abb6ba9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:13:13.117632 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.117589 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gppcq\" (UniqueName: \"kubernetes.io/projected/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-kube-api-access-gppcq\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:13:13.117632 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.117625 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:13:13.117632 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.117641 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e62899-4e17-4b75-8e8e-9193abb6ba9b-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:13:13.813568 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.813523 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" event={"ID":"c1e62899-4e17-4b75-8e8e-9193abb6ba9b","Type":"ContainerDied","Data":"3b540ef9feac37015071a0b6283d6f88b64e94eecced4940d20121a3c6aa657c"} Jul 06 18:13:13.813997 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.813596 2580 scope.go:117] "RemoveContainer" containerID="711784d3b07bba55bbca022e05c1716a173bef0b529d61206eff6a3728c6aba7" Jul 06 18:13:13.813997 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.813549 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl" Jul 06 18:13:13.835333 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.835305 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl"] Jul 06 18:13:13.839141 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:13.839117 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-453d8d-5f9f6bb4cd-lm8hl"] Jul 06 18:13:14.768071 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:14.768033 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e62899-4e17-4b75-8e8e-9193abb6ba9b" path="/var/lib/kubelet/pods/c1e62899-4e17-4b75-8e8e-9193abb6ba9b/volumes" Jul 06 18:13:19.255311 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.255258 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln"] Jul 06 18:13:19.255786 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.255528 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" podUID="4897a06b-50a8-484c-9625-7983bba68c1f" containerName="maas-api" containerID="cri-o://1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d" gracePeriod=30 Jul 06 18:13:19.506273 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.506210 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:19.583121 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.583084 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4897a06b-50a8-484c-9625-7983bba68c1f-openshift-service-ca\") pod \"4897a06b-50a8-484c-9625-7983bba68c1f\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " Jul 06 18:13:19.583354 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.583170 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2h4m\" (UniqueName: \"kubernetes.io/projected/4897a06b-50a8-484c-9625-7983bba68c1f-kube-api-access-d2h4m\") pod \"4897a06b-50a8-484c-9625-7983bba68c1f\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " Jul 06 18:13:19.583354 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.583213 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls\") pod \"4897a06b-50a8-484c-9625-7983bba68c1f\" (UID: \"4897a06b-50a8-484c-9625-7983bba68c1f\") " Jul 06 18:13:19.583560 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.583535 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4897a06b-50a8-484c-9625-7983bba68c1f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4897a06b-50a8-484c-9625-7983bba68c1f" (UID: "4897a06b-50a8-484c-9625-7983bba68c1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:13:19.585272 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.585244 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4897a06b-50a8-484c-9625-7983bba68c1f" (UID: "4897a06b-50a8-484c-9625-7983bba68c1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:13:19.585419 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.585319 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4897a06b-50a8-484c-9625-7983bba68c1f-kube-api-access-d2h4m" (OuterVolumeSpecName: "kube-api-access-d2h4m") pod "4897a06b-50a8-484c-9625-7983bba68c1f" (UID: "4897a06b-50a8-484c-9625-7983bba68c1f"). InnerVolumeSpecName "kube-api-access-d2h4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:13:19.684354 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.684278 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4897a06b-50a8-484c-9625-7983bba68c1f-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:13:19.684354 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.684349 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4897a06b-50a8-484c-9625-7983bba68c1f-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:13:19.684354 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.684360 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2h4m\" (UniqueName: \"kubernetes.io/projected/4897a06b-50a8-484c-9625-7983bba68c1f-kube-api-access-d2h4m\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:13:19.846113 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.846020 2580 generic.go:356] "Generic (PLEG): container finished" podID="4897a06b-50a8-484c-9625-7983bba68c1f" containerID="1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d" exitCode=0 Jul 06 18:13:19.846113 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.846066 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" Jul 06 18:13:19.846113 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.846100 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" event={"ID":"4897a06b-50a8-484c-9625-7983bba68c1f","Type":"ContainerDied","Data":"1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d"} Jul 06 18:13:19.846430 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.846157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln" event={"ID":"4897a06b-50a8-484c-9625-7983bba68c1f","Type":"ContainerDied","Data":"f0e7062c4c67416ec0b766671e000f573554d91a1845c271bdccc20d5f7c0932"} Jul 06 18:13:19.846430 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.846181 2580 scope.go:117] "RemoveContainer" containerID="1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d" Jul 06 18:13:19.856268 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.856253 2580 scope.go:117] "RemoveContainer" containerID="1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d" Jul 06 18:13:19.856555 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:13:19.856535 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d\": container with ID starting with 1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d not found: ID does not exist" containerID="1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d" Jul 06 18:13:19.856663 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.856562 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d"} err="failed to get container status \"1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d\": rpc error: code = NotFound desc = could not find container \"1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d\": container with ID starting with 1cf6d4f2c6d221193acb79bcb3eff105ef1bdadee94221927e01c84bd5b7827d not found: ID does not exist" Jul 06 18:13:19.869327 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.869277 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln"] Jul 06 18:13:19.875063 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:19.875035 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-f63d8f-574fbf4cb8-rsrln"] Jul 06 18:13:20.767978 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:20.767943 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4897a06b-50a8-484c-9625-7983bba68c1f" path="/var/lib/kubelet/pods/4897a06b-50a8-484c-9625-7983bba68c1f/volumes" Jul 06 18:13:26.819350 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:26.819319 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:13:26.823361 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:26.823329 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:13:28.474656 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.474619 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg"] Jul 06 18:13:28.475335 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.475306 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e62899-4e17-4b75-8e8e-9193abb6ba9b" containerName="maas-api" Jul 06 18:13:28.475335 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.475334 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e62899-4e17-4b75-8e8e-9193abb6ba9b" containerName="maas-api" Jul 06 18:13:28.475624 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.475357 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4897a06b-50a8-484c-9625-7983bba68c1f" containerName="maas-api" Jul 06 18:13:28.475624 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.475366 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4897a06b-50a8-484c-9625-7983bba68c1f" containerName="maas-api" Jul 06 18:13:28.475624 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.475482 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e62899-4e17-4b75-8e8e-9193abb6ba9b" containerName="maas-api" Jul 06 18:13:28.475624 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.475493 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4897a06b-50a8-484c-9625-7983bba68c1f" containerName="maas-api" Jul 06 18:13:28.480628 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.480602 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.484365 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.484317 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-3b0a18-serving-cert\"" Jul 06 18:13:28.487884 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.487851 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg"] Jul 06 18:13:28.567310 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.567255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75wfq\" (UniqueName: \"kubernetes.io/projected/f0047c57-40ad-41c6-8ca5-98cff2ea720f-kube-api-access-75wfq\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.567499 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.567400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.567499 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.567454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0047c57-40ad-41c6-8ca5-98cff2ea720f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.668501 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.668463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.668689 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.668514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0047c57-40ad-41c6-8ca5-98cff2ea720f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.668689 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.668565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75wfq\" (UniqueName: \"kubernetes.io/projected/f0047c57-40ad-41c6-8ca5-98cff2ea720f-kube-api-access-75wfq\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.668689 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:13:28.668664 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-3b0a18-serving-cert: secret "maas-api-e2e-shared-a-3b0a18-serving-cert" not found Jul 06 18:13:28.668820 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:13:28.668747 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls podName:f0047c57-40ad-41c6-8ca5-98cff2ea720f nodeName:}" failed. No retries permitted until 2026-07-06 18:13:29.168726635 +0000 UTC m=+2102.919094195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls") pod "maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" (UID: "f0047c57-40ad-41c6-8ca5-98cff2ea720f") : secret "maas-api-e2e-shared-a-3b0a18-serving-cert" not found Jul 06 18:13:28.669247 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.669227 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0047c57-40ad-41c6-8ca5-98cff2ea720f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:28.680556 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:28.680527 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75wfq\" (UniqueName: \"kubernetes.io/projected/f0047c57-40ad-41c6-8ca5-98cff2ea720f-kube-api-access-75wfq\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:29.173463 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:29.173421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:29.175912 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:29.175891 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls\") pod \"maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:29.394184 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:29.394147 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:29.535505 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:29.535461 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg"] Jul 06 18:13:29.538978 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:13:29.538949 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0047c57_40ad_41c6_8ca5_98cff2ea720f.slice/crio-4836011c1af458c90ca1795da0c7250f70ccc3d24de1bc11c03cae119cf886a3 WatchSource:0}: Error finding container 4836011c1af458c90ca1795da0c7250f70ccc3d24de1bc11c03cae119cf886a3: Status 404 returned error can't find the container with id 4836011c1af458c90ca1795da0c7250f70ccc3d24de1bc11c03cae119cf886a3 Jul 06 18:13:29.898635 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:29.898596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" event={"ID":"f0047c57-40ad-41c6-8ca5-98cff2ea720f","Type":"ContainerStarted","Data":"4836011c1af458c90ca1795da0c7250f70ccc3d24de1bc11c03cae119cf886a3"} Jul 06 18:13:30.906269 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:30.906231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" event={"ID":"f0047c57-40ad-41c6-8ca5-98cff2ea720f","Type":"ContainerStarted","Data":"fb7f7e4c5e55186d910a852e16a171b64a35330c7be7708b5a59598152e7e798"} Jul 06 18:13:30.906672 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:30.906323 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:30.926317 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:30.926243 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" podStartSLOduration=2.5240772099999997 podStartE2EDuration="2.92622582s" totalImagesPullingTime="402.14861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:13:30.924629757 +0000 UTC m=+2104.674997332" watchObservedRunningTime="2026-07-06 18:13:30.92622582 +0000 UTC m=+2104.676593396" Jul 06 18:13:36.698594 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.698552 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd"] Jul 06 18:13:36.702522 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.702496 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.707401 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.707361 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-a0e97d-serving-cert\"" Jul 06 18:13:36.716392 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.716361 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd"] Jul 06 18:13:36.846063 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.846000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.846305 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.846114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.846305 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.846184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk7m5\" (UniqueName: \"kubernetes.io/projected/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-kube-api-access-jk7m5\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.917574 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.917548 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:13:36.947578 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.947539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.947795 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.947593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.947795 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.947663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk7m5\" (UniqueName: \"kubernetes.io/projected/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-kube-api-access-jk7m5\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.947795 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:13:36.947733 2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-a0e97d-serving-cert: secret "maas-api-e2e-shared-b-a0e97d-serving-cert" not found Jul 06 18:13:36.947972 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:13:36.947817 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls podName:7ebd8307-e5e3-49d4-b2b9-6594228a3ff3 nodeName:}" failed. No retries permitted until 2026-07-06 18:13:37.447794765 +0000 UTC m=+2111.198162324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls") pod "maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" (UID: "7ebd8307-e5e3-49d4-b2b9-6594228a3ff3") : secret "maas-api-e2e-shared-b-a0e97d-serving-cert" not found Jul 06 18:13:36.948372 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.948332 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:36.958245 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:36.958149 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk7m5\" (UniqueName: \"kubernetes.io/projected/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-kube-api-access-jk7m5\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:37.453023 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:37.452968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:37.455748 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:37.455717 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls\") pod \"maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:37.616774 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:37.616734 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:37.757697 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:37.757671 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd"] Jul 06 18:13:37.760803 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:13:37.760646 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebd8307_e5e3_49d4_b2b9_6594228a3ff3.slice/crio-aca867813c5e8eefef47ec00bb5163bc283671604614e30059920c300b1d034c WatchSource:0}: Error finding container aca867813c5e8eefef47ec00bb5163bc283671604614e30059920c300b1d034c: Status 404 returned error can't find the container with id aca867813c5e8eefef47ec00bb5163bc283671604614e30059920c300b1d034c Jul 06 18:13:37.944197 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:37.944159 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" event={"ID":"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3","Type":"ContainerStarted","Data":"aca867813c5e8eefef47ec00bb5163bc283671604614e30059920c300b1d034c"} Jul 06 18:13:38.950640 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:38.950603 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" event={"ID":"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3","Type":"ContainerStarted","Data":"59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a"} Jul 06 18:13:38.951041 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:38.950683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:13:38.969507 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:38.969450 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" podStartSLOduration=2.564342409 podStartE2EDuration="2.969431923s" totalImagesPullingTime="405.089514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:13:38.967879471 +0000 UTC m=+2112.718247048" watchObservedRunningTime="2026-07-06 18:13:38.969431923 +0000 UTC m=+2112.719799499" Jul 06 18:13:44.960720 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:13:44.960690 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:15:00.132624 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.132587 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd"] Jul 06 18:15:00.135919 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.135896 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" Jul 06 18:15:00.147756 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.147703 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd"] Jul 06 18:15:00.244110 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.244066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4pkh\" (UniqueName: \"kubernetes.io/projected/12700d15-993e-4768-a0e6-ca8868252c9f-kube-api-access-b4pkh\") pod \"maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd\" (UID: \"12700d15-993e-4768-a0e6-ca8868252c9f\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" Jul 06 18:15:00.344752 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.344712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4pkh\" (UniqueName: \"kubernetes.io/projected/12700d15-993e-4768-a0e6-ca8868252c9f-kube-api-access-b4pkh\") pod \"maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd\" (UID: \"12700d15-993e-4768-a0e6-ca8868252c9f\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" Jul 06 18:15:00.353020 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.352987 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4pkh\" (UniqueName: \"kubernetes.io/projected/12700d15-993e-4768-a0e6-ca8868252c9f-kube-api-access-b4pkh\") pod \"maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd\" (UID: \"12700d15-993e-4768-a0e6-ca8868252c9f\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" Jul 06 18:15:00.448982 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.448893 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" Jul 06 18:15:00.584830 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:00.584804 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd"] Jul 06 18:15:00.586878 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:15:00.586851 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12700d15_993e_4768_a0e6_ca8868252c9f.slice/crio-79dfa18c5013cf853d2dc892db055d45d9a1d8a1c852ee8b11dbe16b4aafb029 WatchSource:0}: Error finding container 79dfa18c5013cf853d2dc892db055d45d9a1d8a1c852ee8b11dbe16b4aafb029: Status 404 returned error can't find the container with id 79dfa18c5013cf853d2dc892db055d45d9a1d8a1c852ee8b11dbe16b4aafb029 Jul 06 18:15:01.357085 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:01.357050 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" event={"ID":"12700d15-993e-4768-a0e6-ca8868252c9f","Type":"ContainerStarted","Data":"79dfa18c5013cf853d2dc892db055d45d9a1d8a1c852ee8b11dbe16b4aafb029"} Jul 06 18:15:05.378125 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:05.378084 2580 generic.go:356] "Generic (PLEG): container finished" podID="12700d15-993e-4768-a0e6-ca8868252c9f" containerID="7f188dcba68e950fd708b045a21ac48eff3940b9b52ecf8b47364feba99660c4" exitCode=0 Jul 06 18:15:05.378517 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:05.378166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" event={"ID":"12700d15-993e-4768-a0e6-ca8868252c9f","Type":"ContainerDied","Data":"7f188dcba68e950fd708b045a21ac48eff3940b9b52ecf8b47364feba99660c4"} Jul 06 18:15:06.519507 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:06.519480 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" Jul 06 18:15:06.605785 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:06.605752 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4pkh\" (UniqueName: \"kubernetes.io/projected/12700d15-993e-4768-a0e6-ca8868252c9f-kube-api-access-b4pkh\") pod \"12700d15-993e-4768-a0e6-ca8868252c9f\" (UID: \"12700d15-993e-4768-a0e6-ca8868252c9f\") " Jul 06 18:15:06.607802 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:06.607770 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12700d15-993e-4768-a0e6-ca8868252c9f-kube-api-access-b4pkh" (OuterVolumeSpecName: "kube-api-access-b4pkh") pod "12700d15-993e-4768-a0e6-ca8868252c9f" (UID: "12700d15-993e-4768-a0e6-ca8868252c9f"). InnerVolumeSpecName "kube-api-access-b4pkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:15:06.706590 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:06.706494 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4pkh\" (UniqueName: \"kubernetes.io/projected/12700d15-993e-4768-a0e6-ca8868252c9f-kube-api-access-b4pkh\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:15:07.389447 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:07.389414 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" Jul 06 18:15:07.389603 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:07.389448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd" event={"ID":"12700d15-993e-4768-a0e6-ca8868252c9f","Type":"ContainerDied","Data":"79dfa18c5013cf853d2dc892db055d45d9a1d8a1c852ee8b11dbe16b4aafb029"} Jul 06 18:15:07.389603 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:15:07.389491 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79dfa18c5013cf853d2dc892db055d45d9a1d8a1c852ee8b11dbe16b4aafb029" Jul 06 18:16:57.942967 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:16:57.942934 2580 generic.go:356] "Generic (PLEG): container finished" podID="ba8ba816-0274-411b-bf7f-6269530ff2cb" containerID="ca6c60ad7975a4ff756ccda0a1ec6c66e6b0d1f27563a8364e90259951a62ebb" exitCode=1 Jul 06 18:16:57.943419 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:16:57.943008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" event={"ID":"ba8ba816-0274-411b-bf7f-6269530ff2cb","Type":"ContainerDied","Data":"ca6c60ad7975a4ff756ccda0a1ec6c66e6b0d1f27563a8364e90259951a62ebb"} Jul 06 18:16:57.943419 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:16:57.943386 2580 scope.go:117] "RemoveContainer" containerID="ca6c60ad7975a4ff756ccda0a1ec6c66e6b0d1f27563a8364e90259951a62ebb" Jul 06 18:16:57.943691 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:16:57.943678 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:16:58.952813 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:16:58.952779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" event={"ID":"ba8ba816-0274-411b-bf7f-6269530ff2cb","Type":"ContainerStarted","Data":"305b5cdcfd387b1bd5622bc6a3f6a17dae47a0126ee43a47f176eb0e48d876a7"} Jul 06 18:16:58.953314 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:16:58.953096 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 18:17:09.960165 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:17:09.960136 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-597bcb8998-btmg2" Jul 06 18:18:26.859605 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:18:26.859572 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:18:26.861874 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:18:26.861852 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w5lpr_df4e23f6-8e9f-4390-923e-7c97baffa4e3/ovn-acl-logging/0.log" Jul 06 18:19:03.242024 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.241966 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg"] Jul 06 18:19:03.242794 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.242279 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" podUID="f0047c57-40ad-41c6-8ca5-98cff2ea720f" containerName="maas-api" containerID="cri-o://fb7f7e4c5e55186d910a852e16a171b64a35330c7be7708b5a59598152e7e798" gracePeriod=30 Jul 06 18:19:03.576691 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.576656 2580 generic.go:356] "Generic (PLEG): container finished" podID="f0047c57-40ad-41c6-8ca5-98cff2ea720f" containerID="fb7f7e4c5e55186d910a852e16a171b64a35330c7be7708b5a59598152e7e798" exitCode=0 Jul 06 18:19:03.576852 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.576706 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" event={"ID":"f0047c57-40ad-41c6-8ca5-98cff2ea720f","Type":"ContainerDied","Data":"fb7f7e4c5e55186d910a852e16a171b64a35330c7be7708b5a59598152e7e798"} Jul 06 18:19:03.695876 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.695849 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:19:03.796042 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.795941 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0047c57-40ad-41c6-8ca5-98cff2ea720f-openshift-service-ca\") pod \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " Jul 06 18:19:03.796042 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.796014 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75wfq\" (UniqueName: \"kubernetes.io/projected/f0047c57-40ad-41c6-8ca5-98cff2ea720f-kube-api-access-75wfq\") pod \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " Jul 06 18:19:03.796318 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.796049 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls\") pod \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\" (UID: \"f0047c57-40ad-41c6-8ca5-98cff2ea720f\") " Jul 06 18:19:03.796433 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.796405 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0047c57-40ad-41c6-8ca5-98cff2ea720f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0047c57-40ad-41c6-8ca5-98cff2ea720f" (UID: "f0047c57-40ad-41c6-8ca5-98cff2ea720f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:19:03.798183 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.798154 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0047c57-40ad-41c6-8ca5-98cff2ea720f-kube-api-access-75wfq" (OuterVolumeSpecName: "kube-api-access-75wfq") pod "f0047c57-40ad-41c6-8ca5-98cff2ea720f" (UID: "f0047c57-40ad-41c6-8ca5-98cff2ea720f"). InnerVolumeSpecName "kube-api-access-75wfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:19:03.798285 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.798196 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0047c57-40ad-41c6-8ca5-98cff2ea720f" (UID: "f0047c57-40ad-41c6-8ca5-98cff2ea720f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:19:03.897201 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.897159 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0047c57-40ad-41c6-8ca5-98cff2ea720f-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:19:03.897201 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.897191 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75wfq\" (UniqueName: \"kubernetes.io/projected/f0047c57-40ad-41c6-8ca5-98cff2ea720f-kube-api-access-75wfq\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:19:03.897201 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:03.897202 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0047c57-40ad-41c6-8ca5-98cff2ea720f-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:19:04.582967 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:04.582935 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" Jul 06 18:19:04.583493 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:04.582969 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg" event={"ID":"f0047c57-40ad-41c6-8ca5-98cff2ea720f","Type":"ContainerDied","Data":"4836011c1af458c90ca1795da0c7250f70ccc3d24de1bc11c03cae119cf886a3"} Jul 06 18:19:04.583493 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:04.583033 2580 scope.go:117] "RemoveContainer" containerID="fb7f7e4c5e55186d910a852e16a171b64a35330c7be7708b5a59598152e7e798" Jul 06 18:19:04.603975 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:04.603947 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg"] Jul 06 18:19:04.609574 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:04.609547 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-3b0a18-576c8fd744-m4jjg"] Jul 06 18:19:04.772996 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:04.772957 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0047c57-40ad-41c6-8ca5-98cff2ea720f" path="/var/lib/kubelet/pods/f0047c57-40ad-41c6-8ca5-98cff2ea720f/volumes" Jul 06 18:19:46.171000 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.170960 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd"] Jul 06 18:19:46.171621 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.171270 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" podUID="7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" containerName="maas-api" containerID="cri-o://59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a" gracePeriod=30 Jul 06 18:19:46.197677 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.197642 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd"] Jul 06 18:19:46.199532 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.199507 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a0e97d-29722695-grrxd"] Jul 06 18:19:46.424227 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.424158 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:19:46.482830 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.482794 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-openshift-service-ca\") pod \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " Jul 06 18:19:46.482830 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.482833 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls\") pod \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " Jul 06 18:19:46.483046 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.482868 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk7m5\" (UniqueName: \"kubernetes.io/projected/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-kube-api-access-jk7m5\") pod \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\" (UID: \"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3\") " Jul 06 18:19:46.483236 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.483213 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" (UID: "7ebd8307-e5e3-49d4-b2b9-6594228a3ff3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:19:46.485035 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.485004 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-kube-api-access-jk7m5" (OuterVolumeSpecName: "kube-api-access-jk7m5") pod "7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" (UID: "7ebd8307-e5e3-49d4-b2b9-6594228a3ff3"). InnerVolumeSpecName "kube-api-access-jk7m5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:19:46.485157 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.485051 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" (UID: "7ebd8307-e5e3-49d4-b2b9-6594228a3ff3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:19:46.583683 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.583641 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-openshift-service-ca\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:19:46.583683 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.583676 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-maas-api-tls\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:19:46.583683 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.583685 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk7m5\" (UniqueName: \"kubernetes.io/projected/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3-kube-api-access-jk7m5\") on node \"ip-10-0-138-108.ec2.internal\" DevicePath \"\"" Jul 06 18:19:46.768273 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.768241 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12700d15-993e-4768-a0e6-ca8868252c9f" path="/var/lib/kubelet/pods/12700d15-993e-4768-a0e6-ca8868252c9f/volumes" Jul 06 18:19:46.797468 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.797435 2580 generic.go:356] "Generic (PLEG): container finished" podID="7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" containerID="59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a" exitCode=0 Jul 06 18:19:46.797651 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.797488 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" Jul 06 18:19:46.797651 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.797499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" event={"ID":"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3","Type":"ContainerDied","Data":"59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a"} Jul 06 18:19:46.797651 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.797556 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd" event={"ID":"7ebd8307-e5e3-49d4-b2b9-6594228a3ff3","Type":"ContainerDied","Data":"aca867813c5e8eefef47ec00bb5163bc283671604614e30059920c300b1d034c"} Jul 06 18:19:46.797651 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.797583 2580 scope.go:117] "RemoveContainer" containerID="59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a" Jul 06 18:19:46.806623 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.806606 2580 scope.go:117] "RemoveContainer" containerID="59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a" Jul 06 18:19:46.806897 ip-10-0-138-108 kubenswrapper[2580]: E0706 18:19:46.806875 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a\": container with ID starting with 59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a not found: ID does not exist" containerID="59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a" Jul 06 18:19:46.806967 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.806904 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a"} err="failed to get container status \"59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a\": rpc error: code = NotFound desc = could not find container \"59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a\": container with ID starting with 59566dc10a3892660e61935a5fe26cab32e39ebb15b121037d0bbdd36808b91a not found: ID does not exist" Jul 06 18:19:46.814819 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.814792 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd"] Jul 06 18:19:46.816491 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:46.816470 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-a0e97d-78d76b644-9z2kd"] Jul 06 18:19:48.768906 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:48.768871 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" path="/var/lib/kubelet/pods/7ebd8307-e5e3-49d4-b2b9-6594228a3ff3/volumes" Jul 06 18:19:52.657684 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:52.657649 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cbc7ddf6-9vtj5_90b1f7ea-c93d-405a-95e9-19ad67eb63eb/authorino/0.log" Jul 06 18:19:57.545236 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:57.545203 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7cf57666c-x2md2_02e8ecaa-fd1a-4b54-a03d-f1baf16774ae/maas-api/0.log" Jul 06 18:19:57.894528 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:57.894440 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75789b675b-f7tk2_1531bbb3-ff39-4521-9990-924bb99da441/manager/0.log" Jul 06 18:19:58.388014 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:58.387982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597bcb8998-btmg2_ba8ba816-0274-411b-bf7f-6269530ff2cb/copy-manifests/0.log" Jul 06 18:19:58.394796 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:58.394764 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597bcb8998-btmg2_ba8ba816-0274-411b-bf7f-6269530ff2cb/manager/1.log" Jul 06 18:19:59.927034 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:19:59.926999 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cbc7ddf6-9vtj5_90b1f7ea-c93d-405a-95e9-19ad67eb63eb/authorino/0.log" Jul 06 18:20:00.311005 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:00.310980 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-njtcn_4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3/kuadrant-console-plugin/0.log" Jul 06 18:20:00.563135 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:00.563053 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q2mdt_5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b/manager/0.log" Jul 06 18:20:01.520184 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:01.520143 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf85c8449-qhxjq_4a6aba22-66bb-4428-9717-4f35b33ed960/kube-auth-proxy/0.log" Jul 06 18:20:01.764668 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:01.764641 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-s5dxt_72aee57d-e8f8-49f0-b294-d5750e1081c0/payload-pre-processing/0.log" Jul 06 18:20:02.588196 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:02.588162 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-l99xk_d8661a91-f5b6-42ca-9f89-af1d9ba7fc33/main/0.log" Jul 06 18:20:10.347528 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:10.347496 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-64lqb_085b4edd-6175-4c47-b96f-f079c9da977b/global-pull-secret-syncer/0.log" Jul 06 18:20:10.514205 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:10.514169 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lkf6j_d5f611c0-3ab7-4bd8-9332-06ac4922a8c3/konnectivity-agent/0.log" Jul 06 18:20:10.613846 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:10.613759 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-108.ec2.internal_b7ec4acbf852e2b07ece08ecc1022c7c/haproxy/0.log" Jul 06 18:20:15.859487 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:15.859454 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55cbc7ddf6-9vtj5_90b1f7ea-c93d-405a-95e9-19ad67eb63eb/authorino/0.log" Jul 06 18:20:15.937272 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:15.937241 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-njtcn_4d6a3ec5-a544-43eb-ae9a-3fa8fcfecbf3/kuadrant-console-plugin/0.log" Jul 06 18:20:16.100325 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:16.100214 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q2mdt_5f9d6f6f-f0b4-4194-ae62-7fc4085dc18b/manager/0.log" Jul 06 18:20:18.270833 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.270805 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6d9668478f-bs6fw_80bd5c42-7663-43f2-9e10-9f2f59d630d3/metrics-server/0.log" Jul 06 18:20:18.376421 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.376388 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4rjm5_32d03f1e-5b97-42c3-bf29-02159bc2955d/node-exporter/0.log" Jul 06 18:20:18.431121 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.431093 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4rjm5_32d03f1e-5b97-42c3-bf29-02159bc2955d/kube-rbac-proxy/0.log" Jul 06 18:20:18.478953 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.478930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4rjm5_32d03f1e-5b97-42c3-bf29-02159bc2955d/init-textfile/0.log" Jul 06 18:20:18.636465 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.636438 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-6ll6h_7040cc35-7fab-450e-b259-d22694151513/kube-rbac-proxy-main/0.log" Jul 06 18:20:18.655129 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.655102 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-6ll6h_7040cc35-7fab-450e-b259-d22694151513/kube-rbac-proxy-self/0.log" Jul 06 18:20:18.674415 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.674388 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-6ll6h_7040cc35-7fab-450e-b259-d22694151513/openshift-state-metrics/0.log" Jul 06 18:20:18.707645 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.707622 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_707c78bf-5b26-403c-b11f-5b68285e0ee8/prometheus/0.log" Jul 06 18:20:18.724925 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.724896 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_707c78bf-5b26-403c-b11f-5b68285e0ee8/config-reloader/0.log" Jul 06 18:20:18.756508 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.756437 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_707c78bf-5b26-403c-b11f-5b68285e0ee8/thanos-sidecar/0.log" Jul 06 18:20:18.774105 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.774085 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_707c78bf-5b26-403c-b11f-5b68285e0ee8/kube-rbac-proxy-web/0.log" Jul 06 18:20:18.819275 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.819240 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_707c78bf-5b26-403c-b11f-5b68285e0ee8/kube-rbac-proxy/0.log" Jul 06 18:20:18.838615 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.838591 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_707c78bf-5b26-403c-b11f-5b68285e0ee8/kube-rbac-proxy-thanos/0.log" Jul 06 18:20:18.856312 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.856275 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_707c78bf-5b26-403c-b11f-5b68285e0ee8/init-config-reloader/0.log" Jul 06 18:20:18.879099 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.879081 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-ls7xf_66931001-3fdb-4df8-bc40-a88daa32bdac/prometheus-operator/0.log" Jul 06 18:20:18.894958 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.894937 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-ls7xf_66931001-3fdb-4df8-bc40-a88daa32bdac/kube-rbac-proxy/0.log" Jul 06 18:20:18.913285 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.913262 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-d8p89_f1879a5b-52b9-46d8-abd5-cf2ddd6ffac3/prometheus-operator-admission-webhook/0.log" Jul 06 18:20:18.936226 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.936193 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d7df4f5cc-xswfd_18db26d8-6e39-4e47-a2d2-6c196853c659/telemeter-client/0.log" Jul 06 18:20:18.953681 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.953651 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d7df4f5cc-xswfd_18db26d8-6e39-4e47-a2d2-6c196853c659/reload/0.log" Jul 06 18:20:18.974664 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.974635 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d7df4f5cc-xswfd_18db26d8-6e39-4e47-a2d2-6c196853c659/kube-rbac-proxy/0.log" Jul 06 18:20:18.999047 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:18.999022 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f6c55d98-h2czn_d9b57b0f-9235-4e27-b4e8-993443f6dfb2/thanos-query/0.log" Jul 06 18:20:19.015939 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:19.015866 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f6c55d98-h2czn_d9b57b0f-9235-4e27-b4e8-993443f6dfb2/kube-rbac-proxy-web/0.log" Jul 06 18:20:19.032863 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:19.032838 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f6c55d98-h2czn_d9b57b0f-9235-4e27-b4e8-993443f6dfb2/kube-rbac-proxy/0.log" Jul 06 18:20:19.049954 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:19.049924 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f6c55d98-h2czn_d9b57b0f-9235-4e27-b4e8-993443f6dfb2/prom-label-proxy/0.log" Jul 06 18:20:19.067272 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:19.067247 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f6c55d98-h2czn_d9b57b0f-9235-4e27-b4e8-993443f6dfb2/kube-rbac-proxy-rules/0.log" Jul 06 18:20:19.086262 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:19.086237 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-76f6c55d98-h2czn_d9b57b0f-9235-4e27-b4e8-993443f6dfb2/kube-rbac-proxy-metrics/0.log" Jul 06 18:20:20.491799 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.491753 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g"] Jul 06 18:20:20.492250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492157 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0047c57-40ad-41c6-8ca5-98cff2ea720f" containerName="maas-api" Jul 06 18:20:20.492250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492172 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0047c57-40ad-41c6-8ca5-98cff2ea720f" containerName="maas-api" Jul 06 18:20:20.492250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492185 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12700d15-993e-4768-a0e6-ca8868252c9f" containerName="cleanup" Jul 06 18:20:20.492250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492194 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="12700d15-993e-4768-a0e6-ca8868252c9f" containerName="cleanup" Jul 06 18:20:20.492250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492224 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" containerName="maas-api" Jul 06 18:20:20.492250 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492235 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" containerName="maas-api" Jul 06 18:20:20.492538 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492355 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="12700d15-993e-4768-a0e6-ca8868252c9f" containerName="cleanup" Jul 06 18:20:20.492538 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492367 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0047c57-40ad-41c6-8ca5-98cff2ea720f" containerName="maas-api" Jul 06 18:20:20.492538 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.492376 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ebd8307-e5e3-49d4-b2b9-6594228a3ff3" containerName="maas-api" Jul 06 18:20:20.495217 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.495197 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.497991 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.497970 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9w8k6\"/\"openshift-service-ca.crt\"" Jul 06 18:20:20.499355 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.499336 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9w8k6\"/\"default-dockercfg-s5m7h\"" Jul 06 18:20:20.499355 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.499343 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9w8k6\"/\"kube-root-ca.crt\"" Jul 06 18:20:20.505058 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.505029 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g"] Jul 06 18:20:20.596178 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.596143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-lib-modules\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.596389 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.596229 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-sys\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.596389 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.596265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-podres\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.596389 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.596381 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-proc\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.596507 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.596418 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlsmj\" (UniqueName: \"kubernetes.io/projected/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-kube-api-access-wlsmj\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697442 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlsmj\" (UniqueName: \"kubernetes.io/projected/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-kube-api-access-wlsmj\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697658 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-lib-modules\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697658 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-sys\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697658 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-podres\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697658 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-proc\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697658 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697636 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-sys\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697851 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697673 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-lib-modules\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697851 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-proc\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.697851 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.697744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-podres\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.706417 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.706385 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlsmj\" (UniqueName: \"kubernetes.io/projected/a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7-kube-api-access-wlsmj\") pod \"perf-node-gather-daemonset-jw26g\" (UID: \"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7\") " pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.805488 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.805390 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:20.939451 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.939426 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g"] Jul 06 18:20:20.941073 ip-10-0-138-108 kubenswrapper[2580]: W0706 18:20:20.941045 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda57f1d7f_9fca_406b_98a4_6f3eb2c51cd7.slice/crio-5346af0b535f1942e5838f714e06d77f8b06ed0186e28207d31f377d768d78dc WatchSource:0}: Error finding container 5346af0b535f1942e5838f714e06d77f8b06ed0186e28207d31f377d768d78dc: Status 404 returned error can't find the container with id 5346af0b535f1942e5838f714e06d77f8b06ed0186e28207d31f377d768d78dc Jul 06 18:20:20.968481 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:20.968445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" event={"ID":"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7","Type":"ContainerStarted","Data":"5346af0b535f1942e5838f714e06d77f8b06ed0186e28207d31f377d768d78dc"} Jul 06 18:20:21.393799 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:21.393768 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-dm7mj_ba2ffe7e-2e50-42b5-8dca-7f4f2b119381/download-server/0.log" Jul 06 18:20:21.928561 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:21.928530 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-bbp2c_c72fbb11-2094-4b88-b34c-16075f6be7fc/volume-data-source-validator/0.log" Jul 06 18:20:21.975246 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:21.975200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" event={"ID":"a57f1d7f-9fca-406b-98a4-6f3eb2c51cd7","Type":"ContainerStarted","Data":"8c02326c7cc35e9a08410c359960faf282097195c5ae0a2715981a1554c9cbb6"} Jul 06 18:20:21.975447 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:21.975359 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:21.993040 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:21.992986 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" podStartSLOduration=1.992969821 podStartE2EDuration="1.992969821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:21.991333451 +0000 UTC m=+2515.741701030" watchObservedRunningTime="2026-07-06 18:20:21.992969821 +0000 UTC m=+2515.743337397" Jul 06 18:20:22.900763 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:22.900735 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lkl8n_cc27b535-341b-46dd-9be8-7ce1b75e6e7f/dns/0.log" Jul 06 18:20:22.918528 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:22.918501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lkl8n_cc27b535-341b-46dd-9be8-7ce1b75e6e7f/kube-rbac-proxy/0.log" Jul 06 18:20:22.981920 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:22.981894 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d5x7w_d4e72a07-00c8-4c7e-ab0f-a21b8e0ae931/dns-node-resolver/0.log" Jul 06 18:20:23.563355 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:23.563323 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jlngt_e8eddfc3-a5cb-4dfc-b9c9-9b9e3f2c1ab3/node-ca/0.log" Jul 06 18:20:24.664214 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:24.664186 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf85c8449-qhxjq_4a6aba22-66bb-4428-9717-4f35b33ed960/kube-auth-proxy/0.log" Jul 06 18:20:24.777209 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:24.777162 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-s5dxt_72aee57d-e8f8-49f0-b294-d5750e1081c0/payload-pre-processing/0.log" Jul 06 18:20:25.577954 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:25.577924 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jvl6d_465e675d-31d0-4272-b140-23a5e780df74/serve-healthcheck-canary/0.log" Jul 06 18:20:26.174471 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:26.174440 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vtbpl_d311746f-1f0c-4269-a8dc-ab9b5900a8c9/kube-rbac-proxy/0.log" Jul 06 18:20:26.190533 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:26.190505 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vtbpl_d311746f-1f0c-4269-a8dc-ab9b5900a8c9/exporter/0.log" Jul 06 18:20:26.209129 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:26.209103 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vtbpl_d311746f-1f0c-4269-a8dc-ab9b5900a8c9/extractor/0.log" Jul 06 18:20:27.990905 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:27.990877 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9w8k6/perf-node-gather-daemonset-jw26g" Jul 06 18:20:28.531669 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:28.531637 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7cf57666c-x2md2_02e8ecaa-fd1a-4b54-a03d-f1baf16774ae/maas-api/0.log" Jul 06 18:20:28.672026 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:28.671988 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75789b675b-f7tk2_1531bbb3-ff39-4521-9990-924bb99da441/manager/0.log" Jul 06 18:20:28.858780 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:28.858697 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597bcb8998-btmg2_ba8ba816-0274-411b-bf7f-6269530ff2cb/manager/1.log" Jul 06 18:20:28.896202 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:28.896169 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597bcb8998-btmg2_ba8ba816-0274-411b-bf7f-6269530ff2cb/manager/0.log" Jul 06 18:20:28.910728 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:28.910704 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597bcb8998-btmg2_ba8ba816-0274-411b-bf7f-6269530ff2cb/copy-manifests/0.log" Jul 06 18:20:30.522135 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:30.522103 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68c469dbcb-n4srm_622b26a0-5ac6-43bf-afb5-b4bec79c5981/manager/0.log" Jul 06 18:20:30.580634 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:30.580553 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hllll_d9b6db0c-d1be-492b-b2ff-8581db2597e2/openshift-lws-operator/0.log" Jul 06 18:20:31.479089 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:31.479065 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf85c8449-qhxjq_4a6aba22-66bb-4428-9717-4f35b33ed960/kube-auth-proxy/0.log" Jul 06 18:20:31.573676 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:31.573641 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-s5dxt_72aee57d-e8f8-49f0-b294-d5750e1081c0/payload-pre-processing/0.log" Jul 06 18:20:33.104999 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:33.104964 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf85c8449-qhxjq_4a6aba22-66bb-4428-9717-4f35b33ed960/kube-auth-proxy/0.log" Jul 06 18:20:33.204559 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:33.204522 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-s5dxt_72aee57d-e8f8-49f0-b294-d5750e1081c0/payload-pre-processing/0.log" Jul 06 18:20:36.730270 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:36.730242 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wm6tm_35d04c92-1a0e-42ee-a63e-91dc5e35f966/kube-multus-additional-cni-plugins/0.log" Jul 06 18:20:36.747829 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:36.747801 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wm6tm_35d04c92-1a0e-42ee-a63e-91dc5e35f966/egress-router-binary-copy/0.log" Jul 06 18:20:36.764393 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:36.764353 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wm6tm_35d04c92-1a0e-42ee-a63e-91dc5e35f966/cni-plugins/0.log" Jul 06 18:20:36.780183 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:36.780155 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wm6tm_35d04c92-1a0e-42ee-a63e-91dc5e35f966/bond-cni-plugin/0.log" Jul 06 18:20:36.796691 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:36.796669 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wm6tm_35d04c92-1a0e-42ee-a63e-91dc5e35f966/routeoverride-cni/0.log" Jul 06 18:20:36.812468 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:36.812442 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wm6tm_35d04c92-1a0e-42ee-a63e-91dc5e35f966/whereabouts-cni-bincopy/0.log" Jul 06 18:20:36.828020 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:36.827994 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wm6tm_35d04c92-1a0e-42ee-a63e-91dc5e35f966/whereabouts-cni/0.log" Jul 06 18:20:37.088577 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:37.088519 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qk5dl_cddf9fb1-4a06-4bd4-939a-c9b59ca24f30/kube-multus/0.log" Jul 06 18:20:37.183029 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:37.183000 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nch8h_d2e94037-82c6-43ee-bc44-d391a411b64d/network-metrics-daemon/0.log" Jul 06 18:20:37.197491 ip-10-0-138-108 kubenswrapper[2580]: I0706 18:20:37.197463 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nch8h_d2e94037-82c6-43ee-bc44-d391a411b64d/kube-rbac-proxy/0.log"